-- Logs begin at Tue 2022-09-13 19:05:57 UTC, end at Tue 2022-09-13 20:10:00 UTC. -- Sep 13 19:35:23.306509 dv-cinder-828806-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 19:35:23.315648 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 19:35:23 2022] *** Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: nodename: dv-cinder-828806-6-iscsi Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: machine: x86_64 Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: clock source: unix Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: pcre jit disabled Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: detected number of CPU cores: 4 Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: current working directory: / Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: detected binary path: /usr/bin/uwsgi-core Sep 13 19:35:23.317710 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: your processes number limit is 47680 Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: your memory page size is 4096 bytes Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: detected max file descriptor number: 2048 Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: lock engine: pthread robust mutexes Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: thunder lock: enabled Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 19:35:23.318328 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 19:35:23.356512 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: Python main interpreter initialized at 0x557f7c5639c0 Sep 13 19:35:23.356512 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: python threads support enabled Sep 13 19:35:23.356512 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: your server socket listen backlog is limited to 100 connections Sep 13 19:35:23.356512 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: your mercy for graceful operations on workers is 90 seconds Sep 13 19:35:23.356948 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 19:35:23.356948 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: *** Operational MODE: preforking *** Sep 13 19:35:23.357223 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: *** uWSGI is running in multiple interpreter mode *** Sep 13 19:35:23.357223 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: spawned uWSGI master process (pid: 93078) Sep 13 19:35:23.357453 dv-cinder-828806-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 19:35:23.359351 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: spawned uWSGI worker 1 (pid: 93090, cores: 1) Sep 13 19:35:23.359351 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: spawned uWSGI worker 2 (pid: 93091, cores: 1) Sep 13 19:35:23.359351 dv-cinder-828806-6-iscsi devstack@keystone.service[93078]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 19:35:27.609796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:27.609796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: from cryptography.utils import int_from_bytes Sep 13 19:35:27.610655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:27.610655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: from cryptography.utils import int_from_bytes Sep 13 19:35:27.614099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:27.614099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: from cryptography.utils import int_from_bytes Sep 13 19:35:27.614683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:27.614683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: from cryptography.utils import int_from_bytes Sep 13 19:35:27.659816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:35:27.660069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:35:27.660144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] command line args: [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:35:27.660431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:35:27.660578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ================================================================================ {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:35:27.660861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] admin_token = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.661106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] config_dir = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.661331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.661577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] config_source = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.661706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] debug = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.661786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:35:27.661922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.662179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:35:27.662179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] command line args: [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:35:27.662286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.662350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:35:27.662493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.662574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ================================================================================ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:35:27.662703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.662791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] admin_token = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.662978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_dir = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_config_append = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] config_source = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] debug = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_dir = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_file = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.663902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.664949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_config_append = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_dir = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.665930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.666934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.667910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] publish_errors = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.668976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] standard_threads = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.18/identity {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] publish_errors = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.669967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] use_journal = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] use_json = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] use_stderr = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] use_syslog = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.670930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] standard_threads = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.671449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.671666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.671895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.671895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_journal = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.672099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.672207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_json = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.672322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.672444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_stderr = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.672556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.672665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] use_syslog = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.672780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.672906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:27.673029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.673955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.674975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.675899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.676998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.677534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.677534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.677771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.677882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.677991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.678901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.679914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.680965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.681979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.682949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.683955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.684926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.685907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.686925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.687892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.688975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.689894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.690914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.691938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.692983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.693960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.694883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.695893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.external = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.696934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.password = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] auth.token = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.697930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.external = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.698937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.password = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] auth.token = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.699953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.700974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.701896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.702906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.703941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.704881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.705948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.706972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.707866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.708811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.709851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.710999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.711994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.712859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.713930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.714938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.715948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.716996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.717940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.718953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.719954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.720852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.721969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.722980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.723948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.724921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.725959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.726972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727650 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] role.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.727903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] role.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.728956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.729977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.730964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.731955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.732999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.733905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.734920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.735931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.736850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.737954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.738988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.739910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.connection = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.740929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.741946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742759 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.742983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.743886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93090) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:35:27.744455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.744933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.745074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.745313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.745558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.745789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.746029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.746264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:27.746434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:35:27.773196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.773426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.773649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.775066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.775303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.775532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.782089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.783930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.810372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.810580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.812095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.812340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.864630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.864817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.913518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.913737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.913878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:27.913994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.072979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.073919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.074037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.074116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.074233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.097982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.098860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.109792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.110129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.122511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.122654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.122790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.122878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.122987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93090) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.123980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:28.140388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.140572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:35:28.140973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.141176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:35:28.141302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.141444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:35:28.141525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.141697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.141842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.141842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:35:28.142049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.142183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.142264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.142383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.142672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.142931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.151797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.152050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.152353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.152577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.152705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.152837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.153144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.153432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.159587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.160370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.162569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.162854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.163438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.163990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.164995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.165070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.165295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.165426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.165585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.165830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.166099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.188028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.188615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.189529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.189812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.190116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.190409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.194642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.194922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.195067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.195189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.195414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.195663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.201665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.201955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.202081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.202307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.207998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.208333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.208460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.208623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.208710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.208924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.209939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.210070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.210362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.225525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.225789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.225933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.226111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.229646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.230025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.232831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.233156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.233303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.233573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.237326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.237779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.240408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.240931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.243609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.243917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.244996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.245100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.245311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.245591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.260698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.261011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.261264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.261392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.261532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.261613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.261812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.262083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.268292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.268834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.269835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.270390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.271653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.271918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.272046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.272175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.274636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.274860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.274961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.275812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.276103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.287090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.287325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.287473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.287615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.287698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.287889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.288929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.289111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.304319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.304562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.307288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.307507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.310204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.310426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.311697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.311864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.313204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.313352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.313533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.313620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.313884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.314012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.320331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.320479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.321826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.321954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.324717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.324846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.325851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.333930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.334058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.334225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.334350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.334507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.334659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.341377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.341504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93090) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:35:28.341665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.341793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.341927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.342844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.343939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.344929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.345012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93090) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:35:28.379055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557f7c5639c0 pid: 93091 (default app) Sep 13 19:35:28.379197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x557f7c5639c0 pid: 93090 (default app) Sep 13 19:35:28.382460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e69461-a745-4758-bdc1-410e507717cc None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:28.382751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e69461-a745-4758-bdc1-410e507717cc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:28.383023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e69461-a745-4758-bdc1-410e507717cc None None] PATH_INFO: `/v3` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:28.387193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1/1] 10.222.0.18 () {50 vars in 791 bytes} [Tue Sep 13 19:35:28 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:35.155639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.156225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.156225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.160418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1/2] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:35.166227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:35.166787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:35.166787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:35.238668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: INFO dbcounter [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] Registered counter for database keystone Sep 13 19:35:35.258114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG oslo_db.sqlalchemy.engines [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93091) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:35:35.265619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writer thread running {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:35:35.441997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG passlib.handlers.bcrypt [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93091) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:35:35.442359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG passlib.handlers.bcrypt [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93091) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:35:35.500189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:35.501849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:35.634959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 2/3] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:35:36.116675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:36.186746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: INFO dbcounter [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] Registered counter for database keystone Sep 13 19:35:36.209655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG oslo_db.sqlalchemy.engines [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93090) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:35:36.217397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writer thread running {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:35:36.340901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d28ddf-30fc-4f70-96a4-3ff6afcd415e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:36.345077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:36.345706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:36.345980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:36.346245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:36.354675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:36.404994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:35:36.405337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94fb308-0bb6-499f-8225-a7c62b59d857 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:36.416414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 2/4] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 13 19:35:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:35:36.422499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:36.439762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f835a-5591-42b7-913a-6c0774f958a9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:36.444788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:36.445954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:36.446232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:36.446498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] PATH_INFO: `/v3/projects/af00e3aad1ad462b983823ea889f0033` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:36.448157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] RBAC: Authorizing `identity:get_project(project_id=af00e3aad1ad462b983823ea889f0033)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:36.499878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:35:36.500218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d743ca01-9fce-4e12-956e-d30b96ae956f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:36.504770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 3/5] 10.222.0.18 () {56 vars in 1207 bytes} [Tue Sep 13 19:35:36 2022] GET /identity/v3/projects/af00e3aad1ad462b983823ea889f0033 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:38.629383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad465-af66-4f6d-821c-32af2aa0f607 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.629969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad465-af66-4f6d-821c-32af2aa0f607 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.629969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ad465-af66-4f6d-821c-32af2aa0f607 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.634078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 3/6] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:38.639725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.640166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.640258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.661170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:38.662751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:38.671143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 4/7] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:35:38.934759 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ad465-af66-4f6d-821c-32af2aa0f607 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.948114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ad465-af66-4f6d-821c-32af2aa0f607 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.952913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.953517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.953798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.954064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.962433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.964213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c23b3-11a8-48ff-9eea-0055cf947cc3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:38.973329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 4/8] 10.222.0.18 () {58 vars in 1324 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:35:38.979387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:38.989068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa290a2b-35e9-44f1-9e02-2d3ede0e3bbc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:38.994704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:38.995336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:38.995610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:38.995874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:38.997367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:38.999203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a389d9e0-ec38-48c7-8b45-f5ddf519f585 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:39.000802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 5/9] 10.222.0.18 () {56 vars in 1198 bytes} [Tue Sep 13 19:35:38 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:41.206409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a52e0d-1857-4d01-b793-ccfa86ae6782 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.206993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a52e0d-1857-4d01-b793-ccfa86ae6782 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.206993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a52e0d-1857-4d01-b793-ccfa86ae6782 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.208290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 5/10] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:41.213792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.214249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.214346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.227162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161481-aa96-4584-8602-41793b520f96 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.227418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161481-aa96-4584-8602-41793b520f96 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.227717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1161481-aa96-4584-8602-41793b520f96 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.229030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 6/11] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:41.232979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:41.233985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.234168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:41.234263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.234517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.277131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG passlib.handlers.bcrypt [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93090) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:35:41.277403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG passlib.handlers.bcrypt [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93090) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:35:41.319383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 6/12] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:41.326082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:41.327711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:41.432080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 7/13] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:41.582565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.608528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:41.690457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75628a9-7d77-4b67-a651-02eb7449f085 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.692211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.726961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.727600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.727888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.728159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.739324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:41.748150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:41.749164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:35:41.749998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 7/14] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:41.755204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:41.801149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30f1a2a-ba8b-4bc6-b922-3bcd85a2c823 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.803443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:41.839573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.840096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:41.840375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.840652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.840914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.859547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab41df-6db1-4e6b-bf1e-444dc0933dd7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:41.892670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:41.893326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:41.893609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:41.893881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:41.894574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:41.894874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:41.896284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:41.908254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 8/15] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:41.910152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:41.911683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:41.914125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.021225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4100b-ab98-48d4-8aee-d72a8188b2bd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.023518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 8/16] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:42.029505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.032450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.059502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.060144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.060422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.060692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.061867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.063660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.065560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 9/17] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:41 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:42.070993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.086924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.141158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe83e00-28c5-493f-952d-9b8b44649736 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.170797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc5fc79-f749-4149-81a7-09ae275f8fcd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.173200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.174081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.174365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.174640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.194226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.195725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.205874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.206517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.206792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.207064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.208314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.209804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.211485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 10/18] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:42.214933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:42.216328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:42.217347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:42.217347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:42.217347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:35:42.217347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 9/19] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:42.218739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.222131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:42.277151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.299449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:42.318854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5085f2d-843a-4c87-85cd-5bee65362068 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.323884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd9b1b-5a14-4e1a-b82b-5a881a3b7eea None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:42.352431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.353083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.353361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.353628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.354357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.domain.id=default, user.id=cc92c723f30741629d9940fdc108b101) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:42.354663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.356190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3672374-b19d-4cfd-a9dd-8da8ce95eb68 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.357857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:42.358500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:42.358778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:42.359049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:42.359706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:42.360009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:42.361488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a689adb-360f-47b1-bdf0-4868843d9265 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:42.364075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 11/20] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/role_assignments?user.id=cc92c723f30741629d9940fdc108b101&scope.domain.id=default&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:42.382204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 10/21] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:35:44.796368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3d254-8bd2-4f79-bcf9-26c776623a89 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.796920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3d254-8bd2-4f79-bcf9-26c776623a89 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.796920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3d254-8bd2-4f79-bcf9-26c776623a89 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.798285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 12/22] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:44.803069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668334-0ad6-4e65-9d47-08050b240794 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.803536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668334-0ad6-4e65-9d47-08050b240794 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.803641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e668334-0ad6-4e65-9d47-08050b240794 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.805280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 11/23] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:44.808190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9213-7585-4cc2-9043-51336c40cc56 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.808660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9213-7585-4cc2-9043-51336c40cc56 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.809500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d9213-7585-4cc2-9043-51336c40cc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.810931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05242-defe-4407-b0cc-865c4daace39 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:44.811395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05242-defe-4407-b0cc-865c4daace39 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:44.811843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c05242-defe-4407-b0cc-865c4daace39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:44.842179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2c9d9213-7585-4cc2-9043-51336c40cc56 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:44.843747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2c9d9213-7585-4cc2-9043-51336c40cc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:44.846109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-27c05242-defe-4407-b0cc-865c4daace39 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:44.849512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-27c05242-defe-4407-b0cc-865c4daace39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:44.926404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 12/24] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:44.979176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 13/25] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:45.173400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de54f-8dae-494d-809d-1ede71421063 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.174019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de54f-8dae-494d-809d-1ede71421063 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.174495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1de54f-8dae-494d-809d-1ede71421063 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.176054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 13/26] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:45.181776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.182351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.182815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.208188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:45.209854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:45.248044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de54f-8dae-494d-809d-1ede71421063 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.307287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 14/27] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:45.350096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.423317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1de54f-8dae-494d-809d-1ede71421063 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.423882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.440204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.471256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.471922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.472206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.472475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.487177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.488678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:35:45.529415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:45.530472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:35:45.531271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:35:45.531271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:35:45.531271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:35:45.532778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 14/28] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:35:45.538408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.652526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e501b2-4d7b-4738-acc3-60e11862dce1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.719881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.720195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.720453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.720724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.736716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.742605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b57a02-2072-4b5e-bde4-5ea8ac65ebb0 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.744403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:45.750193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:45.751038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:35:45.752188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 15/29] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:45.758548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.822349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.823045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.823238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.823509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.824161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:45.824465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.826104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.853267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 15/30] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.855022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:45.883637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:45.929815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f08d8-0476-4883-b1c6-2b47acd8fc39 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.973923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:45.975183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:45.975908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:45.976746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:45.977905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:45.978394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:45.979325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7477bde5-0a0d-4a86-9312-a93253c0b70d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:45.980224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:45.992528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 16/31] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:45.994790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:46.032101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.032810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.033168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.033498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.034236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.040084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894674e-4dea-47e8-91a9-6f6be1a5f795 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.123604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff405ed-0126-4ce2-ba8e-6d8c57c4f0df None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:46.152743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:46.163325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.164365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.164809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.165322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.166167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.167812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d93d81-9775-44b4-82bf-158c24b8b695 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.193866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 16/32] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 19:35:45 2022] POST /identity/v3/roles => generated 219 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:46.195732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461b63-9d90-4dce-ab56-9a68256dfde9 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.196139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461b63-9d90-4dce-ab56-9a68256dfde9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.196488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461b63-9d90-4dce-ab56-9a68256dfde9 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.198517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 17/33] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:46.204082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a461b63-9d90-4dce-ab56-9a68256dfde9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:46.226645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:46.324046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 17/34] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:35:45 2022] POST /identity/v3/projects => generated 301 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:46.326045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.326566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.327030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.359183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:46.361226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:46.410304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a461b63-9d90-4dce-ab56-9a68256dfde9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:46.443362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 18/35] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:46.462868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.463568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.463855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.464129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.466084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.467856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.469908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 18/36] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:46 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:46.475838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:46.615477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f168518-7eed-415f-ac76-76849a133e8e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:46.743873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:46.747084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:46.777538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.778590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.779068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.779534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.781035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.783450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423ece0-7e08-46eb-8dcf-0f88cca68396 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.785348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 19/37] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:46.786885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e26c-fdcf-4a1a-9bd8-e4e8ae3ff104 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.787406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e26c-fdcf-4a1a-9bd8-e4e8ae3ff104 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.787864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e26c-fdcf-4a1a-9bd8-e4e8ae3ff104 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.790053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 20/38] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:46.793643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e26c-fdcf-4a1a-9bd8-e4e8ae3ff104 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:46.863179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:46.878306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fbf7b-0edd-4bd6-b68d-49192533ba29 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:46.903105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e26c-fdcf-4a1a-9bd8-e4e8ae3ff104 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:46.910974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.911699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.911978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.912248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.913028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.914439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfccc3c-868d-409b-a15e-5b058116d004 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.930020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 19/39] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 19:35:46 2022] POST /identity/v3/roles => generated 225 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:46.931526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.931960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.932230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.939655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:46.940661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.941035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.941407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] PATH_INFO: `/v3/domains/default/users/cc92c723f30741629d9940fdc108b101/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.943832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc92c723f30741629d9940fdc108b101, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:46.945547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c5e403-7052-4609-88a0-7f09af99731d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:46.954132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:46.955963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:46.974068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 21/40] 10.222.0.18 () {58 vars in 1361 bytes} [Tue Sep 13 19:35:46 2022] PUT /identity/v3/domains/default/users/cc92c723f30741629d9940fdc108b101/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:35:46.974068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3f011-af6f-44c4-8b6f-833f98574ae8 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.974068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3f011-af6f-44c4-8b6f-833f98574ae8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.974068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d3f011-af6f-44c4-8b6f-833f98574ae8 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:46.974068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 22/41] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:46.974686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:46.974890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:46.975167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:47.000847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:47.003026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:47.054821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 20/42] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:47.068556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:47.114919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 23/43] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:47.330343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:47.383551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:47.416337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:47.443876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521dce7e-8ea3-49f8-a41e-cb66c5128c59 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:47.481783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:47.482529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:47.482809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:47.483082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:47.483752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:47.485467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:47.488021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f422c3-eafc-4667-9942-b60e73b72e75 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:47.504022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 21/44] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 19:35:47 2022] POST /identity/v3/roles => generated 223 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:47.532458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:47.533265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:47.533619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:47.533999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:47.535485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:47.537234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3c31f-6716-4a89-989f-d4b07f6d48d2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:47.539318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 24/45] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:47.545240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:47.660670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac058246-9136-4b82-b64e-0fa954fdf4c3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:47.700520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:47.701192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:47.701471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:47.702644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:47.702766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:47.704450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992ecf49-ca8b-41be-ab5d-9a39185bc904 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:47.710755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:47.737457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 22/46] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:35:47 2022] POST /identity/v3/projects => generated 312 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:49.157730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874e7-0d91-4aa5-ba02-0d51721283e6 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.158268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874e7-0d91-4aa5-ba02-0d51721283e6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.158268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510874e7-0d91-4aa5-ba02-0d51721283e6 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.159689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 25/47] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:49.165310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.165761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.165848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.185159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:49.186365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:49.256322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 23/48] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:49.517896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510874e7-0d91-4aa5-ba02-0d51721283e6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.554859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:49.625299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510874e7-0d91-4aa5-ba02-0d51721283e6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.660957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.661604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.661868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.662140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.672284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:49.677284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:49.678193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:35:49.678971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 26/49] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:49.684428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.796233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6507ed-b9bf-46b9-86da-9e021847bf40 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.813739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:49.832074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.832704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.833008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.833280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.833922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:49.834220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:49.835636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f90114-67f9-4cff-9f9d-a9a022bf4381 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:49.847645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 24/50] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:49.853458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:49.879458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f2c2-8084-4155-bb9a-11e9ccf2dae1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.879749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f2c2-8084-4155-bb9a-11e9ccf2dae1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.880056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2f2c2-8084-4155-bb9a-11e9ccf2dae1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.881439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 25/51] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:49.886502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.886781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.887047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.904921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:49.906124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:49.947459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:49.955326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1cc4a-55ad-43e1-9c59-051710c41894 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:49.975143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 26/52] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:49.991407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:49.992047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:49.992326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:49.992597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:49.993777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:49.995241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:49.996831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 27/53] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:49 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.002495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.105315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ff289-0046-42ac-b9c3-e8d1a90b82ec None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.117074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.141995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.142636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.142936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.143228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.144368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.145859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.147398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 27/54] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.153515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.241372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.259318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db626160-0bd8-4a31-8354-b3c8327a7a38 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.295246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.295878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.296158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.296427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.297251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.domain.id=default, user.id=cc92c723f30741629d9940fdc108b101) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:50.297557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.299095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.306116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 28/55] 10.222.0.18 () {56 vars in 1322 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/role_assignments?user.id=cc92c723f30741629d9940fdc108b101&scope.domain.id=default&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.460144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f329429-739f-418c-988f-1d7fd0faf825 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.498481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.499592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.499592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.499860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.500992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.502458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eca0c-4773-4b46-a4bd-d9de1056a79c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.504049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 28/56] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:50.509937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:50.542165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:50.620412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a436f2-4044-4f06-8360-ea3e6cdb2182 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:50.654897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:50.655509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:50.655788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:50.656061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:50.656713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:50.658214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a1627-2aae-489d-8eec-0076d061b495 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:50.686996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 29/57] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 13 19:35:50 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:52.852028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd5ee-190f-4750-a27c-93825c90788a None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:52.852853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd5ee-190f-4750-a27c-93825c90788a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:52.852853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbd5ee-190f-4750-a27c-93825c90788a None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:52.854611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 29/58] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:52.861002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:52.861526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:52.861526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:52.890680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:52.892030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:52.962696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 30/59] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:53.245416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbd5ee-190f-4750-a27c-93825c90788a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:53.295038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:53.353396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbd5ee-190f-4750-a27c-93825c90788a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:53.387461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:53.388086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.388367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.388633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:53.389795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:53.391338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994ccdc-11c6-44d9-b107-a42f872a28cf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:53.392868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 30/60] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:53.398740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:53.409057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:53.503993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575650d7-ac81-4842-8d2a-bfacd56f1923 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:53.554993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:53.555608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:53.555881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:53.556149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:53.556800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:53.558310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e500a8a-04a2-462b-88e1-79b5d631c4db None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:53.591139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 31/61] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:53 2022] POST /identity/v3/users => generated 275 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:56.869111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a54837-24f1-4cbe-aa83-e53279a8dc1a None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.870138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a54837-24f1-4cbe-aa83-e53279a8dc1a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.870690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a54837-24f1-4cbe-aa83-e53279a8dc1a None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.872268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 31/62] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:56.878113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2f89-27fd-482e-a8e1-78b823df2733 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.878971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2f89-27fd-482e-a8e1-78b823df2733 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.879440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d2f89-27fd-482e-a8e1-78b823df2733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.904352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-953d2f89-27fd-482e-a8e1-78b823df2733 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:56.905995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-953d2f89-27fd-482e-a8e1-78b823df2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:56.934568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3d144-0638-4e25-89fc-30f7dbbcd9de None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.935200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3d144-0638-4e25-89fc-30f7dbbcd9de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.935725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3d144-0638-4e25-89fc-30f7dbbcd9de None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.937376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 32/63] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:35:56.942682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b58e4-c0db-4691-8fb7-a741fe68484f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:56.943225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b58e4-c0db-4691-8fb7-a741fe68484f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:56.943692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3b58e4-c0db-4691-8fb7-a741fe68484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:56.976257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bc3b58e4-c0db-4691-8fb7-a741fe68484f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:56.977850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bc3b58e4-c0db-4691-8fb7-a741fe68484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:57.010474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.027971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 32/64] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:57.100928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 33/65] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:35:57.215765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634595e-c2c2-474a-a911-d1aa80796a53 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.215955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634595e-c2c2-474a-a911-d1aa80796a53 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.216230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634595e-c2c2-474a-a911-d1aa80796a53 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.217694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 33/66] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:57.223297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.223584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.223862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.254046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:57.255273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:57.307237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.343651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 34/67] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:57.376381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634595e-c2c2-474a-a911-d1aa80796a53 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.494106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.501701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634595e-c2c2-474a-a911-d1aa80796a53 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.537966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.538970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.539447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.540020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.551277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-acc99969-a00e-4770-90ee-81377e20d4f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:57.557012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:57.557869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:35:57.558627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:35:57.561005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 34/68] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:57.567425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15900-ac4b-4b2f-9380-994e1089ee38 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.568045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15900-ac4b-4b2f-9380-994e1089ee38 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.568549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f15900-ac4b-4b2f-9380-994e1089ee38 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.570868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 35/69] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:57.571830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f15900-ac4b-4b2f-9380-994e1089ee38 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.636999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.638139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92542b0-f59d-4914-ad88-903da63126a8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.679552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.680297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.680511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.680782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.695324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f15900-ac4b-4b2f-9380-994e1089ee38 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.695832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-768f32eb-da8c-4f70-8b25-f2b5c2b3395d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:57.702720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:57.703645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:35:57.704439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 35/70] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:57.706513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.706513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.706513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.725426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:57.726748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:57.742552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.743201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.743488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.743750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.744473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:57.744721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.746162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.755103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 36/71] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:57.755866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.764743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.811872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 36/72] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:57.812805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.858369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0886d07-3557-4c3d-b699-4a44e60d16ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.874908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:57.893868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.894517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.894958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.895425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.905906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:57.911206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:57.912064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 37/73] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:57.913815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:57.925229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b574fcb-f181-4241-aaea-30a00ed9f923 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:57.962117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:57.962974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.963432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.963719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.964380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:57.964680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:57.966705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a55499-cc92-4d3f-9a05-55448d2be38e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:57.980007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 37/74] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:57.981477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eec8-2ce0-4c98-b666-74b6ea244045 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:57.981810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eec8-2ce0-4c98-b666-74b6ea244045 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:57.982108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4eec8-2ce0-4c98-b666-74b6ea244045 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:57.983494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 38/75] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:35:57.984108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4eec8-2ce0-4c98-b666-74b6ea244045 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.014316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.024091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c9bef-681d-4746-ab68-ad5b7025449d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.060939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.061644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.062026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.062370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.063659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.065247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.067229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 38/76] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.067843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.099995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4eec8-2ce0-4c98-b666-74b6ea244045 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.114088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.134492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.135110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.135391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.135670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.136365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.136664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.138145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bbfe7-69a6-497e-9922-d0bc0d8e47bf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.147684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 39/77] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:35:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.148978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.149266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.149529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.167349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:35:58.168584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:35:58.171513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79017947-7a7c-4049-a972-fc75b8e99b6d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.205021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.205666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.205946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.206220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.220338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.221855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.223512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 39/78] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.224290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.234966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 40/79] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:35:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:35:58.235746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.258449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.331480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611dcf-232c-4920-860e-9e4a1525b5ba None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.333250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.347491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4aac8-b59f-4805-acd1-af32e16fc811 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.369237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.370238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.370807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.371085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.381358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.382005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.382292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.382576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.393634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:35:58.398511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:35:58.399346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:35:58.400111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 41/80] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:35:58.400731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.446400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.448018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.485665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.508341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bf9b1-a819-4fec-bc22-814a5330a6fc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.543719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.544577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.544863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.545173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.546458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.547962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.549618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 42/81] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.550428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.562559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 40/82] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:58.563972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.567330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.651782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f5de9-fac3-42cb-a11c-a956a9be79d7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.674047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a7176e-f205-48ee-94ab-7f40ccc78a9c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.692787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.693437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.693720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.693994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.708124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.708737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.709129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.709336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.710146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:58.710262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.710337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.711821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.721265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 41/83] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.722086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.764718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.766248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.784436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.823904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbebad4c-0844-4de4-8b67-dcbac29570ec None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.857395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:58.858017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:58.858288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:58.858556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:58.859645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:58.861158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:58.862732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 42/84] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:58.867404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.867508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 43/85] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:58.867508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:58.914392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:58.970756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c45d0c-4e8f-491b-b93b-4aaffa3df581 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:58.972036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62610-7096-4733-8c8e-0b7888071e6a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.002370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.005950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.006607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.006891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.007224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.007971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.008378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.008577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.008852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.009149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.009893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.011594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 44/86] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.012365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.080381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.081879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.109658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3761-8740-4c51-b761-d80a4968ac47 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.123678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.144056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.144693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.145002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.145270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.146377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.147830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.149464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 45/87] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.150200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.178988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 43/88] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:59.179763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.211408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.252913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a916c95-eca8-4337-ac9f-afefcaad17a9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.280500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cae6e-2422-4f70-9b85-7be4e68665e7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.287953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.288993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.289275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.289546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.290203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.291647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.316432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.317133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.317414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.317689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.319161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.320372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 46/89] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:35:59 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:35:59.328036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.328131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.328244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 44/90] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.328244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.371884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.447804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b56ede-f86d-4f5c-8af4-6c5ce1c12ee8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.474037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.478749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfd7aa-5c42-4629-b42f-c8c49eb40c4b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.488254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.488977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.489199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.489473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.490307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=cc92c723f30741629d9940fdc108b101) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.490624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.492145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.501615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 47/91] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=cc92c723f30741629d9940fdc108b101&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.502562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.523664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.524273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.524548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.524818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.633744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.635254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.652387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641775a3-c56f-4120-864e-cc99dda0b971 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.655401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.703262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.703847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.704130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.704397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.706104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.709584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.709712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 48/92] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.710417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.771533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 45/93] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:35:59.773172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.797316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:35:59.821390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df3faf-bc48-4ddf-89db-79d2747e25db None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.863049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:35:59.863670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:35:59.863946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:35:59.864216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:35:59.864969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:35:59.865276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:35:59.866735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:35:59.875148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 49/94] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:35:59.875994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:35:59.961130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64e774-c354-4d4b-9888-12bada7f4320 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:35:59.976408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:00.049450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:00.049450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.049450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.049450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.052122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:00.052122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:00.055370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:00.074479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 46/95] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b62430e4550a43989b6adfc24920894e => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:00.162657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d3e97-7f73-4147-bffa-1a72d10457dc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:00.207776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:00.208443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.208726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.209033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] PATH_INFO: `/v3/projects/cf41373725124f2f91b77a4d5026e82f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.210266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] RBAC: Authorizing `identity:get_project(project_id=cf41373725124f2f91b77a4d5026e82f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:00.211892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be5d9fe-4066-43fa-8530-045e20a3761f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:00.213708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 50/96] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:35:59 2022] GET /identity/v3/projects/cf41373725124f2f91b77a4d5026e82f => generated 312 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:00.220015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:00.288955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:00.353565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9e946-d0ac-42d8-a0bd-205d1046d622 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:00.418230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:00.418853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.419133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.419404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.420133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=cf41373725124f2f91b77a4d5026e82f, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:00.420438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:00.422025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc7479-a80a-4856-b176-a5abd603cff7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:00.428750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 47/97] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:00 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=cf41373725124f2f91b77a4d5026e82f&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:02.327742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e1a-0d2b-42ab-a210-a842579ea842 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.328269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e1a-0d2b-42ab-a210-a842579ea842 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.328269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe6e1a-0d2b-42ab-a210-a842579ea842 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.329658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 51/98] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.339938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e9aa0-35b1-440b-a689-eea0d29fd911 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.340474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e9aa0-35b1-440b-a689-eea0d29fd911 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.340474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84e9aa0-35b1-440b-a689-eea0d29fd911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.370308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c84e9aa0-35b1-440b-a689-eea0d29fd911 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.371569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c84e9aa0-35b1-440b-a689-eea0d29fd911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.464078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 48/99] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:02.471667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca7b-dba9-4f0c-859c-33796bb41e22 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.471946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca7b-dba9-4f0c-859c-33796bb41e22 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.472226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430ca7b-dba9-4f0c-859c-33796bb41e22 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.473613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 52/100] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.479327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc8c1-076f-4ccb-94b6-14dda78c2c6f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.479918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc8c1-076f-4ccb-94b6-14dda78c2c6f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.480375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fc8c1-076f-4ccb-94b6-14dda78c2c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.502472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b62fc8c1-076f-4ccb-94b6-14dda78c2c6f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.503674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b62fc8c1-076f-4ccb-94b6-14dda78c2c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.576735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:02.592292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 49/101] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:02.791252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cbb9a-7dbb-4f1d-8832-c86eb891880c None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.791621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cbb9a-7dbb-4f1d-8832-c86eb891880c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.791793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7cbb9a-7dbb-4f1d-8832-c86eb891880c None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.793417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 53/102] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:02.798885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:02.799173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:02.799454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:02.824717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:02.825935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:02.921656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 50/103] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:02.948481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cbb9a-7dbb-4f1d-8832-c86eb891880c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.095322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7cbb9a-7dbb-4f1d-8832-c86eb891880c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.122047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.128924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.143274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.143943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.144232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.144511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.163737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:03.176370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:03.178069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 54/104] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.183758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.392525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53d001-6925-4c32-9adb-02a8fa15b6a7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.434813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.436173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c833501-6630-4d20-93fc-b207790b2b16 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.458722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.459530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.459934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.460211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.473947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ebd8140c-3566-4660-bb06-3ac212d7ae02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:03.481957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:03.483060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:03.484012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:03.486109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 51/105] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:03.487560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a959b5-731c-45a3-a1d8-9b201bc31c76 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.488682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a959b5-731c-45a3-a1d8-9b201bc31c76 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.489187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a959b5-731c-45a3-a1d8-9b201bc31c76 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.490670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 52/106] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:03.491555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a959b5-731c-45a3-a1d8-9b201bc31c76 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.495464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.496112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.496426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.499995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.499995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.499995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.500260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.511304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 55/107] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:03.512662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.543800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.624863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a959b5-731c-45a3-a1d8-9b201bc31c76 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.642404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512f0e-fab3-4f97-95ae-09a1b19a2c8d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.661222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.661859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.662148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.662405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.663642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.664998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab27a7b2-7955-43ca-bb65-0c75d35f8480 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.666884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 53/108] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.670855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8d35-da36-4478-a72a-68f16c71f77b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.671538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8d35-da36-4478-a72a-68f16c71f77b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.672138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb8d35-da36-4478-a72a-68f16c71f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.678503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.679123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.679404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.679678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.680403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:03.680722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.682181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.690213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3ebb8d35-da36-4478-a72a-68f16c71f77b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:03.695906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 56/109] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.696068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3ebb8d35-da36-4478-a72a-68f16c71f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:03.696146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.737340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 54/110] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce5dbb-7af9-459d-a62e-ecd0249a19e0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce5dbb-7af9-459d-a62e-ecd0249a19e0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce5dbb-7af9-459d-a62e-ecd0249a19e0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 55/111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:03.772909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce5dbb-7af9-459d-a62e-ecd0249a19e0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.805808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:03.826118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a814d-d4a7-48f3-b367-bcc89654f06c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.863760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.864401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.864799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.865099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.866331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.867769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.869930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 57/112] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:03.870773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:03.885011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce5dbb-7af9-459d-a62e-ecd0249a19e0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:03.917944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:03.918561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.918841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.919112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.919760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:03.921226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf7119-ab45-4ec3-b33e-9fabd6238b07 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:03.949978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 56/113] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:03 2022] POST /identity/v3/projects => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:03.951134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43cc13-1820-438a-bc20-77c70d928102 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:03.951490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43cc13-1820-438a-bc20-77c70d928102 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:03.951962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea43cc13-1820-438a-bc20-77c70d928102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:03.972636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ea43cc13-1820-438a-bc20-77c70d928102 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:03.974067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ea43cc13-1820-438a-bc20-77c70d928102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:03.986731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ea59a-fa33-4d85-8963-662a81832ccf None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.008828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.037254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.038377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.038936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.039406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] PATH_INFO: `/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.054365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 57/114] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:04.058901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] RBAC: Authorizing `identity:get_user(user_id=be911fd000ce4f64a1741e3a8f3ecf3c)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.059392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c33ab-d0db-4412-b189-4d35bc9628f9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.059747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c33ab-d0db-4412-b189-4d35bc9628f9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.060095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c33ab-d0db-4412-b189-4d35bc9628f9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.061442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.061601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 58/115] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:04.062518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c33ab-d0db-4412-b189-4d35bc9628f9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.063861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 58/116] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:03 2022] GET /identity/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.064902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.081379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.185757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2000451-6518-4bbf-89fc-8568d6c9f738 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.194024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c33ab-d0db-4412-b189-4d35bc9628f9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.232240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.232808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.233131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.233401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.235812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.236556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.241765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.241765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.241765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.241765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.241765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 59/117] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.242687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.251871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ba3bc1d6-8234-486f-9f19-c79e352bc7cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:04.257250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.258368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:04.259365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:04.260762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 59/118] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.261964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.262300 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.262646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.275952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.283834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:04.285271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:04.361554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a44b78f-e04f-40a0-9995-e27c2a5c1410 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.380653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 60/119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:04.389526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.389526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.397339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.398263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.398547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.398819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.484593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.486559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.508255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1179d5b-517e-4e82-bde2-3fb39144a5c6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.521847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.549264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.550216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.550639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.550916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/cc92c723f30741629d9940fdc108b101/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.552814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, user_id=cc92c723f30741629d9940fdc108b101, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.554294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.572668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 61/120] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:04 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/cc92c723f30741629d9940fdc108b101/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:04.576037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.617717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 60/121] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:04.618673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.667789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.729374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c3abe-3dcb-44a8-ac52-4d0f4ed0ef2e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.735430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cdefe-a81c-4a4e-83df-856a441705f4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.767423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.779566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.780327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.780665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.780971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.781259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.782334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.782811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.783559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.784968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:04.785496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.787483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.794097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:04.799276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.800185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:04.800968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 61/122] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.801595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.803068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 62/123] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.804137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.920862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d3d5d-17d1-4f29-b186-b678f39a8c3f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.926818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c622e46-15e6-4c08-bb04-5bb0369dbe68 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:04.949824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:04.964710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.965691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.966162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.966824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.972701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:04.973583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:04.974054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:04.974436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:04.975648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:04.977854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:04.979605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 62/124] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:04.981722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:04.985074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:04.990933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:04.991766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:04.992527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:04.994969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 63/125] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:04.996012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.066341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.114628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464ab39-6ccf-4888-aac2-4767e8956c21 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.116972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c383364-19de-4e2e-b3fe-ccf91b65adfb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.148776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.149771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.149878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.150068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.152758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.154269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.155291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.155879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 64/126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.156315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.156791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.157290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.158142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.158629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.160178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.161515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.170317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 63/127] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.171415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.221381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.290185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfa09a-b93f-4397-9404-d194dd2e5032 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.307441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a498299-6271-4901-9607-66ad468d0394 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.329070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.336739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.338165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.338590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.339263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.340160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=be911fd000ce4f64a1741e3a8f3ecf3c) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.340902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.343249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.351311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 65/128] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/role_assignments?user.id=be911fd000ce4f64a1741e3a8f3ecf3c&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b646c18b962b467d83b73b821972238a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.352478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.363167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.364193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.367208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.367780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.368750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:05.369307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.375214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.392230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 64/129] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.393611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.531347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2083bb-2a47-4a5f-a142-c31b97ce3e07 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.549784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.550998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb5c56-84ab-442a-9240-776ec1cfe8e9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.571360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.572336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.572753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.573208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] PATH_INFO: `/v3/projects/cf41373725124f2f91b77a4d5026e82f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.574609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] RBAC: Authorizing `identity:get_project(project_id=cf41373725124f2f91b77a4d5026e82f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.576320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.578344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 66/130] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/projects/cf41373725124f2f91b77a4d5026e82f => generated 312 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.583241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.599357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.600019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.600302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.600575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.605637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.607143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.608926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 65/131] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.609684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.662024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.698720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd4b89-0c59-4e81-bd5e-e690e65281ed None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.732990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca5198d-117a-4a6c-8d19-15d8a10a7a5f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.734565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.735387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.735763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.736833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.737664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.739205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.748913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 67/132] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.748913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.797621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.798014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:05.798582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.799228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.799228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] PATH_INFO: `/v3/projects/cf41373725124f2f91b77a4d5026e82f/users/51e799da9248476094c6ad78ce4252ce/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.801242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] RBAC: Authorizing `identity:create_grant(project_id=cf41373725124f2f91b77a4d5026e82f, user_id=51e799da9248476094c6ad78ce4252ce, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.802640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.850641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01c935-87e7-48a6-bd61-40503c97a8a3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:05.856906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 66/133] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:05 2022] PUT /identity/v3/projects/cf41373725124f2f91b77a4d5026e82f/users/51e799da9248476094c6ad78ce4252ce/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:05.856906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:05.943640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:05.944316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:05.944603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:05.944897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:05.946274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:05.947971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:05.955597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 68/134] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:05.976702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.109439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd8fa-41e5-40f9-bca3-8145074d7d3d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.122395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.180644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.181345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.181625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.181896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.183428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.184929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.187087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 67/135] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:05 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:06.193655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:06.194568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b5dfd7-6d8f-4be7-8f93-8f79335ec4a7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.245563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:06.279779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.280487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.280770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.281077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/51e799da9248476094c6ad78ce4252ce/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.288427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, user_id=51e799da9248476094c6ad78ce4252ce, role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.290098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f375b-5c01-436c-880a-0488adff6beb None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.315333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 69/136] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:05 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/51e799da9248476094c6ad78ce4252ce/roles/b62430e4550a43989b6adfc24920894e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:06.385689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3e05a-37a3-4e22-a42f-0f7cd1514d61 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:06.401997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ec582-49a5-482a-a8bc-264a72800fd7 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.402326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ec582-49a5-482a-a8bc-264a72800fd7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.402556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ec582-49a5-482a-a8bc-264a72800fd7 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.403915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 70/137] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:06.415521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.415817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.416132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.446107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:06.446908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:06.447197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:06.447472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/51e799da9248476094c6ad78ce4252ce/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:06.449485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:06.450737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:06.454504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, user_id=51e799da9248476094c6ad78ce4252ce, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:06.456058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:06.479488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 68/138] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:06 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/51e799da9248476094c6ad78ce4252ce/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:06.577332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 71/139] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:06.996841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:07.044937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:07.131822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0a79b-0a9d-4f01-ad08-e33e979d431b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.176382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.177057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.177342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.177623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.181612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.183095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6027c33c-2ebb-47fc-aaed-eaf13048227a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.184726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 69/140] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:07.194016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:07.343632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfefbe-764c-4e04-b153-175585dcd683 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:07.358340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:07.362554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2665-dcac-406d-8199-ba3d3434adae None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.362910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2665-dcac-406d-8199-ba3d3434adae None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.363134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db2665-dcac-406d-8199-ba3d3434adae None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.364696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 70/141] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:07.374449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.375113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.375581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.405145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:07.405784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.406064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.406332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.406999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:07.408509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:07.427578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:07.428825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:07.471161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 72/142] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:07 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:07.543251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 71/143] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:07.931957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.088229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c15383-1bc8-4666-ae82-168a85972fd2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.141664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.142661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.143125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.143580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.157337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:08.163037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:08.163967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:08.164750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:08.166817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 73/144] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:08.172967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.231033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.325474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14e1fc-ebb0-476a-b5db-dcf0e97ffe5b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.369494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.370136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.370394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.370667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.371343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:08.371643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.373058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263844b-ac14-4192-87b8-dbb300159325 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.382928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 72/145] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:08.388553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.459711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.534263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e8a5e-cdf8-4256-b21f-36c4855a8dd0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.534556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e8a5e-cdf8-4256-b21f-36c4855a8dd0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.534742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e8a5e-cdf8-4256-b21f-36c4855a8dd0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.536099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 73/146] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:08.541445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.541727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.542000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.566414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e525b1-7b38-40b6-8a01-697a7843a071 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.581476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:08.582707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:08.630532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.631196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.631475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.631849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] PATH_INFO: `/v3/users/cc92c723f30741629d9940fdc108b101` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.636328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] RBAC: Authorizing `identity:get_user(user_id=cc92c723f30741629d9940fdc108b101)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.639181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.640839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 74/147] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/users/cc92c723f30741629d9940fdc108b101 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:08.646716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.732217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 74/148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:08.780855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d0701-c4df-439c-8d2a-f32b0d93e751 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.782439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:08.825406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.826044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.826324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.826597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.907357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.908971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.017944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.021196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 75/149] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:09.027095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.055502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.167847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35d4b7-ff80-4017-9b1c-89dcf9e87b72 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.226647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.227297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.227584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.227855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.229029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.230504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.232146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 75/150] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:09.238131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.258543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.410553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41348306-3cbf-49fc-bdc8-aff4d8cb49eb None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.419981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.431032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe855a-9714-471a-a130-b7c06bc9ff64 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.473571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.474236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.474508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.474779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.477675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.478773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.479044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.479310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.480054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=cc92c723f30741629d9940fdc108b101) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:09.480543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.482400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b9d5f-d593-4eff-94dc-239b98a713c5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.487209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:09.494024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 76/151] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/role_assignments?user.id=cc92c723f30741629d9940fdc108b101&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:09.495352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722013-9e0b-4cf4-802d-699b7455d50e None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:09.495680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:09.496984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:09.497814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:09.498425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722013-9e0b-4cf4-802d-699b7455d50e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.498425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5722013-9e0b-4cf4-802d-699b7455d50e None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.498425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 77/153] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:09.498949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 76/153] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:09.503970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.504289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.504591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.505183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.541000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:09.546777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:09.686352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:09.695435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0251c45c-17aa-4c0c-9a8a-d5caaa449d9c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.707239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 78/154] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:09.758331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:09.758963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.759239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.759508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.760187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:09.760485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:09.761929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c95994-0275-4b3d-bd4c-e43424297ae4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:09.776195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 77/155] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:09.782146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:09.887041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d4c7-55f6-49d2-8d3b-022e645a7647 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.887372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d4c7-55f6-49d2-8d3b-022e645a7647 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.887584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431d4c7-55f6-49d2-8d3b-022e645a7647 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.889004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 78/156] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:09.906751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:09.907067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:09.907293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:09.936777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:09.938736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:09.948116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2df1-5976-4ee4-943c-76b91c91e3c6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:09.953618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.043522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.044246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.044486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.044732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] PATH_INFO: `/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.047234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] RBAC: Authorizing `identity:get_user(user_id=be911fd000ce4f64a1741e3a8f3ecf3c)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.048722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.050471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 79/157] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:09 2022] GET /identity/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c => generated 289 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.056782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.088789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 79/158] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:10.197845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.258152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054ef91-1f62-45b1-b44c-e6e41ab5a458 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.283945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.304425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.305105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.305383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.305652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.306815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.308380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.310850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 80/159] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.317461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.461086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963248c8-4721-4deb-916b-91b8e338ad72 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.514796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24087ca1-88f7-410f-aa1b-ed7e8e8415b7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.515809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.517091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.517579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.518057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.544847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-eb9bc4fb-9325-40ed-9058-fcfb6fca1c98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:10.555723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.556695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:10.557501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:10.564223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 80/160] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:10.567786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.570263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04be27-9398-452e-8680-b6d843f0434b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.572487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04be27-9398-452e-8680-b6d843f0434b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.572487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04be27-9398-452e-8680-b6d843f0434b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.574054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 81/161] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:10.576588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04be27-9398-452e-8680-b6d843f0434b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.582801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.597725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.598784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.599259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.599821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/be911fd000ce4f64a1741e3a8f3ecf3c/roles/b646c18b962b467d83b73b821972238a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.602140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, user_id=be911fd000ce4f64a1741e3a8f3ecf3c, role_id=b646c18b962b467d83b73b821972238a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.604521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.630526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 81/162] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:10 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/users/be911fd000ce4f64a1741e3a8f3ecf3c/roles/b646c18b962b467d83b73b821972238a => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:10.631879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.752512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04be27-9398-452e-8680-b6d843f0434b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.763238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a819ae1-7c94-45f3-a42f-c1567310c99b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.803922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.805021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.805613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.806087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.807124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:10.807695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.809619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076b9d8-3c48-4357-a693-03ce45faa5fa None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.820228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 82/163] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:10.821976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.822519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.822984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.827191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.827864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.828142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.828408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.849681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:10.855153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:10.856078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:10.856865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 82/164] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:10.857806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.859409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:10.861207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:10.936733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.990596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:10.995078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 83/165] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:11.002157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.082722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7e26f-3c2d-4003-b844-102cab56043f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.134439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.135234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.135610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.135997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.138751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.140342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.142601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 83/166] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.148639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.251891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609d673-1496-4c77-832d-d4790d55e9de None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.325768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.326422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.326700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.326973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.327664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:11.327957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.329422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.333933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daff64c-b62f-47aa-8810-b46550e3be47 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.344089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 84/167] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:11.350018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.402385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.411465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.412151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.412620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.412795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.448982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.544929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.546413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.577591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30da9a9-576c-4972-8ff1-0f6a71dd9295 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.645809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:11.646509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:11.646787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:11.647054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:11.648208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:11.649696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:11.651555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 85/168] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:11.652154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.758183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 84/169] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:11.760014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:11.873907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:11.955117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee1dc44-c185-4736-99a0-9f5bfc693d2d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:11.967470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.053651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.054324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.054611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.054878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.086454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:12.091755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:12.092800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:12.093732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:12.100976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 86/170] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 441 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:12.102003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.126373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c4f7e-6407-4715-9b4b-d4de630c2422 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.225998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4964159-45cd-4671-a139-5309442fecac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.226634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4964159-45cd-4671-a139-5309442fecac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.226909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4964159-45cd-4671-a139-5309442fecac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.227176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4964159-45cd-4671-a139-5309442fecac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.303845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01076ce2-f76e-434a-8fe9-f140fa6220d8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.395528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.396120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.396402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.396673 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] PATH_INFO: `/v3/projects/cf41373725124f2f91b77a4d5026e82f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.398534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] RBAC: Authorizing `identity:get_project(project_id=cf41373725124f2f91b77a4d5026e82f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.400006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.401876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 87/171] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/projects/cf41373725124f2f91b77a4d5026e82f => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.405916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.413266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4964159-45cd-4671-a139-5309442fecac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.414776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4964159-45cd-4671-a139-5309442fecac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.473281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.473475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.573886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 85/172] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 814 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:12.577450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0070d8-99fd-4e2a-af3c-459f0c7d45e6 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.577761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0070d8-99fd-4e2a-af3c-459f0c7d45e6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.578331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0070d8-99fd-4e2a-af3c-459f0c7d45e6 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.582461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 86/173] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:12.587573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0070d8-99fd-4e2a-af3c-459f0c7d45e6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.621776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6d577-b763-4476-bd75-c0c4c481c57e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.693441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.694367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.694831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.695285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.696130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.696601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.698202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.717676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 88/174] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.718994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:12.756995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0070d8-99fd-4e2a-af3c-459f0c7d45e6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.801322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.803067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.803560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.804042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.805012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=cf41373725124f2f91b77a4d5026e82f, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:12.805500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.808278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60184d-877d-4b30-a4c7-eceff13a1f9a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.823079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 87/175] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=cf41373725124f2f91b77a4d5026e82f&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.824993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fc18-46ac-4a70-96fe-be12acad1f28 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.825500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fc18-46ac-4a70-96fe-be12acad1f28 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.825937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56fc18-46ac-4a70-96fe-be12acad1f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.833456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.866109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8b56fc18-46ac-4a70-96fe-be12acad1f28 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:12.867343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8b56fc18-46ac-4a70-96fe-be12acad1f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:12.873875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:12.885844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469c6406-7c2e-48a5-bb65-0b908bbbdd33 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:12.945451 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:12.946027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.946306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.946576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.947756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:12.949260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c9c426-3a4e-4992-811b-a44a40d2b9a5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:12.950989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 89/176] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:12.952598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f230f8-99ac-4c10-96ba-d9eb1422dd0c None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:12.953701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f230f8-99ac-4c10-96ba-d9eb1422dd0c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:12.953701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f230f8-99ac-4c10-96ba-d9eb1422dd0c None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:12.955554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 90/177] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:12.955554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f230f8-99ac-4c10-96ba-d9eb1422dd0c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.007161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 88/178] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:13.015636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12310-5270-495b-a9a4-d797dab99e9c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.015981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12310-5270-495b-a9a4-d797dab99e9c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.016515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c12310-5270-495b-a9a4-d797dab99e9c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.025266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 89/179] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:13.027814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c12310-5270-495b-a9a4-d797dab99e9c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.117401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f230f8-99ac-4c10-96ba-d9eb1422dd0c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.173750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.174687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.174963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.175239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] PATH_INFO: `/v3/users/51e799da9248476094c6ad78ce4252ce` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.176456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] RBAC: Authorizing `identity:get_user(user_id=51e799da9248476094c6ad78ce4252ce)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.178036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6032fd59-3df2-4de1-9f27-d8b78f8b5413 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.179760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 91/180] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:12 2022] GET /identity/v3/users/51e799da9248476094c6ad78ce4252ce => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.181450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.181727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.182061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.182206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c12310-5270-495b-a9a4-d797dab99e9c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.206330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:13.208103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:13.213664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.224509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.225151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.225424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.225735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.226461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.226911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.228619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840a948-9593-4546-8802-fa9330d7d657 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.235639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 90/181] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b62430e4550a43989b6adfc24920894e => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.236615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.236993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.237445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.238385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.262338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:13.263516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:13.285557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 92/182] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:13.286921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.389695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 91/183] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:13.455066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea709-1f4d-4376-bb35-576e7c2dbbec None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.492460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.493031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.493307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.493571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.526445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.575208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.576662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.583568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.767234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 93/184] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:13.768371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.772724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6823e19-1414-4f1c-8604-dc8b3f4b7b7e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.804761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.829938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.834010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.834491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.834921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.858106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:13.859876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:13.863996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.864915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:13.865709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 92/185] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.870898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.904955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d84db-971f-4117-9efd-538375ed3d70 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.938312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.938967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.939239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.939514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.949886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:13.954662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.955552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:13.956437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 94/186] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.957113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.976824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8a945-5fa2-4fb0-befe-23ebf1d05934 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.010619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.011263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.011543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.011868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.013039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.014518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.016206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 93/187] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.016988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.059091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991fe11-fa80-4258-a337-2888efb1c813 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.070445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.082719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.094122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.094758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.095030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.095293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.096387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.097953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1fbd0-0255-4d93-8fca-5f69c55270f8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.099519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 95/188] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.101340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544c600-af20-4788-b126-170eaa42d031 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.101340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544c600-af20-4788-b126-170eaa42d031 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.101340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4544c600-af20-4788-b126-170eaa42d031 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.102607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 96/189] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:14.103175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544c600-af20-4788-b126-170eaa42d031 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.129369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f894e-ff6c-41b9-b19e-8dd608607fb6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.163089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.163703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.163978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.164250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.164940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.165241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.166632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.175318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 94/190] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.176360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.212702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4544c600-af20-4788-b126-170eaa42d031 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.247468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.248180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.248457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.248723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.249429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.249729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.251136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.264629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 97/191] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.264629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.277095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8b9f-ba4f-4e28-b625-0365327239b8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.287840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.314728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.315326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.315447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.315614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.315878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.316802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=51e799da9248476094c6ad78ce4252ce) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.317173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.319021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f5375-0013-4c29-b15f-436809043acb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.325548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 95/192] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/role_assignments?user.id=51e799da9248476094c6ad78ce4252ce&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.326846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.327127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.327394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.355437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:14.357191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:14.371524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f3d8-4a9e-49ec-8ce6-7b9205a2ef0b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.409282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.409929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.410320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.410617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.411287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.412825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.438817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 98/193] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:14 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:14.438817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.452155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 96/194] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:14.453264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.633180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dee09-3c3c-4aa9-ae12-44c4f154e645 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.637266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b394d67-1936-4c11-969d-b74db9f11ea0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.640993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.647595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.679431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.680578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.680787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.681315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.681518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.681797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.682061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.682352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.698059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.700042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.702721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.703196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 97/195] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.708922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.710987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 99/196] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:14.711897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.718923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.819992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75444c67-9106-4eae-b524-95b6d67584b0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.826924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb70eb2-cfc3-4d2c-bbe0-f8b445effdaa None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.863171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.863797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.864072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.864338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.880959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.881565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.881836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.882101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.902112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.925804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.949134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.950608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.972843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.975041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.062856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 98/197] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:15.063951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.101341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 100/198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:15.102216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.126979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.167942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282ed14-9f81-4d6c-b8dd-ef2979ef8f36 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.188534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.203846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.204821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.205316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.205796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.223618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:15.230769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:15.231670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:15.232440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 99/199] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:15.233145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.235297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3611b4-7a2e-44b1-9684-a3376a78c34c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.273760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67896db-e710-4f66-9836-758c4584802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.274804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67896db-e710-4f66-9836-758c4584802c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.275267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67896db-e710-4f66-9836-758c4584802c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.275823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67896db-e710-4f66-9836-758c4584802c None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.277469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67896db-e710-4f66-9836-758c4584802c None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.279201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67896db-e710-4f66-9836-758c4584802c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.281196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 101/200] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.282172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67896db-e710-4f66-9836-758c4584802c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.362549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c7d7-161e-48c5-9a0f-aa761f145240 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.389400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.403123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.404110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.404318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67896db-e710-4f66-9836-758c4584802c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.404776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.405286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.406648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.408499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.410791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 100/201] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.411772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.450849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.451851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.452317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.452798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.453714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.454338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.456532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.458429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.468407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 102/202] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.469568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.552136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934131d-8a37-483a-b3f0-47bc832d0001 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.584076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d922ba2e-1f2c-409b-9059-b97549edd038 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.599677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.600568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.600852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.601152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.601880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=8d0bf2419957495fa3b84dd7b8663bbc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.602189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.604313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.611728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 101/203] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/role_assignments?user.id=8d0bf2419957495fa3b84dd7b8663bbc&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b62430e4550a43989b6adfc24920894e => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.612352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.622484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.623484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.623972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.624426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.625509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=8d0bf2419957495fa3b84dd7b8663bbc) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:15.626016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.627681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.634093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 103/204] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/role_assignments?user.id=8d0bf2419957495fa3b84dd7b8663bbc&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.681409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.735490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529c21c-6c1f-4d99-8a2d-66add94f3e69 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.772807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.773488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.773772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.774039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] PATH_INFO: `/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:15.775313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=be911fd000ce4f64a1741e3a8f3ecf3c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:15.776829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:15.778961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 102/205] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/users/be911fd000ce4f64a1741e3a8f3ecf3c => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:15.784486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:15.892003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:15.939510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39acc2-ad2b-45e7-931b-054965265c60 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:15.981231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:15.982321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:15.982826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:15.983305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.092590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.094221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.180494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.230299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 104/206] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:16.236206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:16.345871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8019f-6ead-48d1-ba03-0cdc291c0fe3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:16.368305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.384353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.385019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.385299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.385567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.386686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.388128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d8e857-f6df-4ba7-bac1-748492a2b16d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.392601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 103/207] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:16.398927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:16.516046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0fa58-8cbc-45ac-932c-90aff7175a15 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:16.556146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:16.556817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.557143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.557440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.558218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, scope.project.id=99ea55dfca19477d991d0bd252ed5c98, user.id=be911fd000ce4f64a1741e3a8f3ecf3c) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:16.558547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:16.560443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf989c1-ae7d-4e8e-aca6-0453a9acbcd7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:16.569605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 105/208] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:16 2022] GET /identity/v3/role_assignments?user.id=be911fd000ce4f64a1741e3a8f3ecf3c&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b646c18b962b467d83b73b821972238a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:17.185167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8fd1-a3e4-432b-8f42-cecc28390d6f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.185681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8fd1-a3e4-432b-8f42-cecc28390d6f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.185681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb8fd1-a3e4-432b-8f42-cecc28390d6f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.187323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 104/209] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:17.193062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81987f-4fe9-4016-bcc3-015f84aead7c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.193508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81987f-4fe9-4016-bcc3-015f84aead7c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.193583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81987f-4fe9-4016-bcc3-015f84aead7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.216405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ec81987f-4fe9-4016-bcc3-015f84aead7c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:17.217654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ec81987f-4fe9-4016-bcc3-015f84aead7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:17.240505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.308510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 106/210] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:17.386719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5b15-e5b0-48d7-b417-8af32af18ddf None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.387026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5b15-e5b0-48d7-b417-8af32af18ddf None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.387274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0b5b15-e5b0-48d7-b417-8af32af18ddf None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.388926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 105/211] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:17.394052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.394336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.394604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.412717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:17.413928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:17.478211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 107/212] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:17.589374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5b15-e5b0-48d7-b417-8af32af18ddf None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.694263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0b5b15-e5b0-48d7-b417-8af32af18ddf None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.729755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.730808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.731273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.731766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.743102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:17.743343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:17.748936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:17.749847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:17.750648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:17.753498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 106/213] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:17.761024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.829141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.851164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.872501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0e89e-3414-49ad-82d7-28e6c858d695 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.900425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db544f12-d30d-451b-82a5-0f08dbe0b6a7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.925615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.926911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.927795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.928753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.930374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.932738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.936177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 108/214] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:17.940345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.942115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.942892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.943726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.944206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.945166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:17.946045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.948871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.962155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 107/215] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:17.969761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.085768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9d1fe-b290-4581-81da-4ad71ae67a65 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.087602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f5292-56b9-493e-a97a-be4bebbea2da None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.099032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.125289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.125480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.125825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.125938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.126105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.126198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.126357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] PATH_INFO: `/v3/users/3b76c2f8af864ca0ac89ca60767e8d56` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.126448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.127023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.128535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971049e4-922e-4f0d-8a5b-907e42232f60 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.144049 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] RBAC: Authorizing `identity:get_user(user_id=3b76c2f8af864ca0ac89ca60767e8d56)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.146100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82b275-ef1b-4d27-9a85-ae7ebc4bec7c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.148152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 108/216] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/users/3b76c2f8af864ca0ac89ca60767e8d56 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.149476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820494b1-4c50-43f1-aca7-884ee9907a65 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.149755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820494b1-4c50-43f1-aca7-884ee9907a65 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.150022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820494b1-4c50-43f1-aca7-884ee9907a65 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.152950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 109/217] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.154834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820494b1-4c50-43f1-aca7-884ee9907a65 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.160785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 109/218] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:18.171339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5f766-0341-4395-90f9-24db24f41045 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.171774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5f766-0341-4395-90f9-24db24f41045 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.172150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c5f766-0341-4395-90f9-24db24f41045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.199178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b3c5f766-0341-4395-90f9-24db24f41045 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.200924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b3c5f766-0341-4395-90f9-24db24f41045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.204507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.282070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820494b1-4c50-43f1-aca7-884ee9907a65 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.294149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 110/219] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:18.302517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e06d3-2f7b-45fa-b86b-509071ff5cf1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.302892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e06d3-2f7b-45fa-b86b-509071ff5cf1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.303053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e06d3-2f7b-45fa-b86b-509071ff5cf1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.308811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 111/220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.310708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.310982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.311243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.335558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.336934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.337549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.337656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.337912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.338187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.386058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.421870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 112/221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:18.430989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.432512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.535516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 110/222] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:18.540262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.566855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.633910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.654169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.673748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cef278-9493-4e09-aa6c-5328b6834452 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.708092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7500d33-3986-4a01-a745-56aa5ad396ec None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.720706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.721394 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.721709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.722051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.723398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.725020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.726786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 113/223] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.727549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.757654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.758663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.758997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.759528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.774268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:18.781135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:18.782040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:18.782839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:18.783520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 111/224] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:18.784915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.843758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6d25e-98e0-4499-aa5f-480468e7f56e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.890668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.891542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.891822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.892328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.896408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.901753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a9d8c-2b77-415d-af32-c65b8e92eb7a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.911051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:18.916450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:18.917627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 114/225] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:18.918618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.942917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.950090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.951448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.952171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.952634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.953793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=3b76c2f8af864ca0ac89ca60767e8d56) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:18.954626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.957067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.965949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 112/226] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/role_assignments?user.id=3b76c2f8af864ca0ac89ca60767e8d56&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.969777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.127634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5abae4-97b2-4772-9e6d-9b5f75acd905 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.182273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d763a8a3-7ec1-471e-bfe2-0da0e88bc2a9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.189098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.189874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.190156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.190442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.191121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:19.191418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.192817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.202189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 115/227] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.208071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.269319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.277708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.278347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.278628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.278899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.279580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:19.279884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.281336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.295442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 113/228] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.301342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.340425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c548386-047e-45a6-8c12-bc8f3f0b229c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.404557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b167697-c131-47f8-9994-31388daece4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.405235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b167697-c131-47f8-9994-31388daece4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.405513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b167697-c131-47f8-9994-31388daece4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.405781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b167697-c131-47f8-9994-31388daece4d None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.407226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.408221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b167697-c131-47f8-9994-31388daece4d None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.411042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b167697-c131-47f8-9994-31388daece4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.414018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 116/229] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:19.419521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b167697-c131-47f8-9994-31388daece4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.459837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e32c1-29c9-4e81-9267-3c8a9a0e46da None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.505595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.506249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.506527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.506803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.508262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.510976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.512811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 114/230] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:19.534353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.612384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b167697-c131-47f8-9994-31388daece4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.658234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.665031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3f3f-f379-4b44-ae64-8a616c900f80 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.675855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.676498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.676904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.677173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.680240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.688906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.688906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 117/231] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.696816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.708874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.731767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.732344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.732623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.732921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.737680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.739126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.740879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 115/232] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.747190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.844761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceea046f-0a6d-42f0-b695-674ff2864164 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.879009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.879694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.879971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.880239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/users/8d0bf2419957495fa3b84dd7b8663bbc/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.882257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, user_id=8d0bf2419957495fa3b84dd7b8663bbc, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.883769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfde6f-76c1-428e-85f6-d6d9ed72c7f3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.903275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 118/233] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:19 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/users/8d0bf2419957495fa3b84dd7b8663bbc/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:19.917612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06bd003-d75a-4381-a557-e0ac167c7e14 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.955676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.956360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.956701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.957011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/users/8d0bf2419957495fa3b84dd7b8663bbc/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.959380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, user_id=8d0bf2419957495fa3b84dd7b8663bbc, role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.960862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb4dc6-3d32-4f20-bd68-fcd3aa2734c8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.976956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 116/234] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:19 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/users/8d0bf2419957495fa3b84dd7b8663bbc/roles/b62430e4550a43989b6adfc24920894e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:20.816253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1682-0a01-4c47-853a-89b4d97987cb None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.816804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1682-0a01-4c47-853a-89b4d97987cb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.816804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb1682-0a01-4c47-853a-89b4d97987cb None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.818565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 119/235] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:20.824077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.824625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.824625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.849920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:20.851131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:20.914086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:20.927452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c611-3b23-4a01-b788-88a4094947da None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.927785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c611-3b23-4a01-b788-88a4094947da None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.927993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193c611-3b23-4a01-b788-88a4094947da None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.929648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 120/236] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:20.943365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 117/237] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:20.945364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.945643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.945911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.968866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:20.970108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:21.054143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 121/238] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:21.055004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.322974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.391786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.430302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ffd1bb-1488-4f8f-bb09-0208e46017e1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.468244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.468899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.469336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.469644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.470999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.472449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.474890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 118/239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.480308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.545248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.627681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c0352-e05d-411c-91d2-146a923627dc None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.655350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de226-0d24-4d1d-ab46-f8bc522e3893 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.704719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.705402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.705681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.705952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.728971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.729600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.729877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.730146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.730819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.732293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86afaa6-48c0-4f7f-af11-c1035391e992 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.735911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:21.740851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:21.741740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:21.742525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:21.746924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 122/240] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:21.747839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.774790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 119/241] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:21 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:21.781546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2546-50e5-49a5-b08c-3a40ac04167e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.781862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2546-50e5-49a5-b08c-3a40ac04167e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.784051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa2546-50e5-49a5-b08c-3a40ac04167e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.787964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 120/242] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:21 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:21.795840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.796159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.797092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.840963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:21.843948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:21.849611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.971769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b25eef-4415-4dd1-aefe-d4c45be68c83 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.998284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 121/243] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:22.033022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.033614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.033892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.034163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.034846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:22.035148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.036553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.046600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 123/244] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.052714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.104346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.175477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f00e7bb-1cd4-4677-924a-fdcacce5cc63 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.217714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.218426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.218705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.218974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] PATH_INFO: `/v3/users/1d832541a1454ccbb27567c9c720f370` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.233311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] RBAC: Authorizing `identity:get_user(user_id=1d832541a1454ccbb27567c9c720f370)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.234712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.236762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 122/245] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/users/1d832541a1454ccbb27567c9c720f370 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.242531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.321402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.368401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f3192-1bd9-4e32-9831-0d25374a1149 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.380284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.414660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.415392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.415670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.415984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.448001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.458552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae54c03-445e-4a90-9ee9-483582a7127f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.511280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.511905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.512235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.512509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.517400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.518961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eba44a-c5f6-4ec8-acd2-31647f0464e7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.532339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:22.540948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:22.541851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:22.542627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:22.543724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 123/246] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:22.549840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.665569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 124/247] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:22.667855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c303-012b-46bb-8a85-cd9238fe2250 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.667855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c303-012b-46bb-8a85-cd9238fe2250 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.667855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41c303-012b-46bb-8a85-cd9238fe2250 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.668701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 125/248] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:22.674085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41c303-012b-46bb-8a85-cd9238fe2250 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.686310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.697954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29797514-1902-48af-a802-fbee731aaa5e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.736764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.737421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.737986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.737986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.738682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:22.739051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.740499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd0166-bf08-4424-b5b9-ba7251c3d360 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.749942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 124/249] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:22.751249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb40-ed18-4f4c-8b12-fce976880ad3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.751534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb40-ed18-4f4c-8b12-fce976880ad3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.751806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044eb40-ed18-4f4c-8b12-fce976880ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.775942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6044eb40-ed18-4f4c-8b12-fce976880ad3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:22.777147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6044eb40-ed18-4f4c-8b12-fce976880ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:22.789665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.813935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41c303-012b-46bb-8a85-cd9238fe2250 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.851447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.852090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.852279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.852756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.854129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.855760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.857676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 126/250] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.858682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.877080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 125/251] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:22.879089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e620fc-168c-43f7-8562-e9790e1bc0a9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.879577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e620fc-168c-43f7-8562-e9790e1bc0a9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.880070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e620fc-168c-43f7-8562-e9790e1bc0a9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.881919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 126/252] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:22.882874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e620fc-168c-43f7-8562-e9790e1bc0a9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.960382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.989987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7a3cd-4706-45f1-8d26-1b431e258141 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.027971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.028575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.028858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.029162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] PATH_INFO: `/v3/users/3b76c2f8af864ca0ac89ca60767e8d56` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.030508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] RBAC: Authorizing `identity:get_user(user_id=3b76c2f8af864ca0ac89ca60767e8d56)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.032044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7acf8-5250-4b48-b8d8-7469d05d0ffb None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.034394 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 127/253] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/users/3b76c2f8af864ca0ac89ca60767e8d56 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:23.035862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.036143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.036416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.056776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:23.057374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e620fc-168c-43f7-8562-e9790e1bc0a9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.058360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:23.135454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.135454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.135454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.135454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.135873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=1d832541a1454ccbb27567c9c720f370) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:23.135873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.136251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.152902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 127/254] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/role_assignments?user.id=1d832541a1454ccbb27567c9c720f370&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b646c18b962b467d83b73b821972238a => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:23.155085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.187036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.199968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 128/255] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:23.304867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b172e-4e48-4929-a44c-fb7c572b73a4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.347075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.347737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.348019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.348289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.349737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.351519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.353609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 128/256] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:23.362111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.414015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.501329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.548573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bed5f7-05ea-4af9-8057-4560110cf6ff None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.598005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.603672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.604647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.604948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.605225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/users/3b76c2f8af864ca0ac89ca60767e8d56/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.609125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, user_id=3b76c2f8af864ca0ac89ca60767e8d56, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:23.612349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:23.628564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 129/257] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:23 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/users/3b76c2f8af864ca0ac89ca60767e8d56/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:23.630146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.647982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd0927-10fd-4634-8241-d3961fad9695 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.741753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.742427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.742706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.742972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.758736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562870a8-f410-4988-8a87-0a73a881e9c6 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:23.785649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:23.791632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:23.792533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:23.793847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 129/258] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:23.797352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:23.801950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.806532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:23.809516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:23.810056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:23.810574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:23.843522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:23.849598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:23.850567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:23.851470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 130/259] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:23.882817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:23.995056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985c3a4-efec-4cde-94ce-ff4fb5488c78 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.054288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1fcdf-6fb9-4288-955b-bc8e49ff4d58 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.095526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789c7c-1160-4527-953c-df248982742d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.096170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789c7c-1160-4527-953c-df248982742d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.096441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789c7c-1160-4527-953c-df248982742d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.096707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1789c7c-1160-4527-953c-df248982742d None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.097418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789c7c-1160-4527-953c-df248982742d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:24.097715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789c7c-1160-4527-953c-df248982742d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.099138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1789c7c-1160-4527-953c-df248982742d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.101899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.102464 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.102739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.103004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.103681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:24.103976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.105414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.118078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.123214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 131/260] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.125242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 130/261] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.129965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789c7c-1160-4527-953c-df248982742d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.151150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.228525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.308127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1789c7c-1160-4527-953c-df248982742d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.371648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.372216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.372490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.372756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.377829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.379404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.381033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 132/262] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.387491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.442264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc4a87-41b6-4ee6-8336-a688709a9ee1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.508672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.509358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.509643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.509901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] PATH_INFO: `/v3/users/8d0bf2419957495fa3b84dd7b8663bbc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.511097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] RBAC: Authorizing `identity:get_user(user_id=8d0bf2419957495fa3b84dd7b8663bbc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.512557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.519658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 131/263] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/users/8d0bf2419957495fa3b84dd7b8663bbc => generated 283 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:24.520589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.590811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f436d0b7-57d6-41f7-93d1-5d87bb95d3b1 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.637027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.642530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:24.673700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.674456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.674734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.675000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.702474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c30e7-d530-4b7f-b3d6-6c3567d199bb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:24.770634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:24.771842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.772324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.772819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.828910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.828910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:24.899366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:24.900771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.026094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.056911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 133/264] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:25.065870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.090809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 132/265] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:25.100928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.102694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.253298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4804d4-9121-43a8-a2a5-09b44a2db0cc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.350497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2be256-8514-43d1-b313-7b3b15bbfce3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.369704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.370410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.370692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.370965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.375613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.375613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.376030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 134/266] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:25.389562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.422680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.423631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.424129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.424570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.425923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.432909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.435660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 133/267] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:25.441920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.504163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.565299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.575102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678c1fa-3ff1-4df2-9431-ece307979e13 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.584628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c46e452-b295-4d26-89f1-626afd130692 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.613181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.614145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.614610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.615130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.616039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=8d0bf2419957495fa3b84dd7b8663bbc) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.616530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.618204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e188d3-71bf-4a63-b1a1-0c23f4eae3e6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.632916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.632916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.632916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.632916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.633360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=8d0bf2419957495fa3b84dd7b8663bbc) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:25.633360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.633360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44eee75-606a-400e-b51f-ce9c32c41403 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.633967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 135/268] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/role_assignments?user.id=8d0bf2419957495fa3b84dd7b8663bbc&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b62430e4550a43989b6adfc24920894e => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.636561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f8269-f5dc-4b9c-bd53-6fcfe3860368 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.637051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 134/269] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:25 2022] GET /identity/v3/role_assignments?user.id=8d0bf2419957495fa3b84dd7b8663bbc&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.637157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f8269-f5dc-4b9c-bd53-6fcfe3860368 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.637476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f8269-f5dc-4b9c-bd53-6fcfe3860368 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.638908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1bb9-9028-487a-81c7-5e9aec6f9aa2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.639360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1bb9-9028-487a-81c7-5e9aec6f9aa2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.639818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f1bb9-9028-487a-81c7-5e9aec6f9aa2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.641240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 135/271] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:25.643197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 136/271] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:25.644520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.645013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.645438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.666101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:25.667732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:25.677063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.677349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.677766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.708289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:25.713172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:25.799801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 136/272] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:25.854392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 137/273] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:26.202503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.232703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.240608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.321928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06c456-453a-4e1a-9ef1-f7eeef5e823b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.369493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.373567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.374320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.374594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.375380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.389560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:26.389755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b50122-d807-4311-88a5-69b223f8ff9c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:26.395466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:26.396381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:26.397216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:26.401600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 137/274] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:26.403540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.441525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.442608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.444442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.445129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.447026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.449428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.451468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 138/275] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:26.456642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.517178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.529898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db11e76-bd53-4d64-9a04-a4b7c3838e6c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.568395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.569053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.569241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2bfd-a59e-473b-8370-bae98a8cc3c9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.569361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.569593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.570263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:26.570602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.572265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b7d74-8bcc-47b8-897a-45f381b4d491 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.584271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 138/276] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:26.586607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c8cc4-f081-4ef1-a8ec-6e24780c8d10 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.587122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c8cc4-f081-4ef1-a8ec-6e24780c8d10 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.587821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27c8cc4-f081-4ef1-a8ec-6e24780c8d10 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.589855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 139/277] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:26.592431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c8cc4-f081-4ef1-a8ec-6e24780c8d10 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.614458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.615357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.615510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.616020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.616987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.619223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db644702-8aaa-4074-9133-b24bbfa0bf3a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.652610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 139/278] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:26 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:26.652610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab487f-0036-4cc6-98a2-2005d4029c1c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.652610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab487f-0036-4cc6-98a2-2005d4029c1c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.652610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab487f-0036-4cc6-98a2-2005d4029c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.669912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d3ab487f-0036-4cc6-98a2-2005d4029c1c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:26.671191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d3ab487f-0036-4cc6-98a2-2005d4029c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:26.673528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.730812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27c8cc4-f081-4ef1-a8ec-6e24780c8d10 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.760936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 140/279] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:26.760936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e08-36fd-40e6-a9dc-eaa2d668b062 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.760936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e08-36fd-40e6-a9dc-eaa2d668b062 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.760936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a431e08-36fd-40e6-a9dc-eaa2d668b062 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.762119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 141/280] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:26.774648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.774952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.775362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.775710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.776125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.776730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.777235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] PATH_INFO: `/v3/users/3b501cae74c946a28ff6b231f2c68d93` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.793547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] RBAC: Authorizing `identity:get_user(user_id=3b501cae74c946a28ff6b231f2c68d93)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.795341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.797415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 140/281] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/users/3b501cae74c946a28ff6b231f2c68d93 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:26.802360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:26.802360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:26.804040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:26.819380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:26.901439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 142/282] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:26.935583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaa6f7b-df54-4fc1-b137-7c78e22d91c9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:26.973255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:26.973875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:26.974149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:26.974419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:26.975145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, user.id=3b501cae74c946a28ff6b231f2c68d93, scope.system=all) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:26.975446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:26.976918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:26.982930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 141/283] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 19:36:26 2022] GET /identity/v3/role_assignments?user.id=3b501cae74c946a28ff6b231f2c68d93&scope.system=all&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:27.056109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.145753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.180251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.184701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b309d-a191-46e2-8043-8a82212926e4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.225438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.226132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.226404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.226682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.240588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:27.245694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.247041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:27.248151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 143/284] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:27.253212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.294142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.340504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8c620-8969-40a2-8268-52238e5d99ab None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.389529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322ccef-9463-4b50-9672-385e1ce1a211 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.405338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.406272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.406692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.407216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.418262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.427314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.427929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.429110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.429389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.430071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.430735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.432125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.437250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:27.443360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:27.444248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:27.445109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 142/285] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:27.451754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.453753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 144/286] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:27.461437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.619284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a717ed82-aa2a-42c7-b1a9-85a4badf1c0d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.648569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836c117-cc42-4e34-b6a3-ec53d1ce218b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.705441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.706462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.706884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.707297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.708616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:27.709226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.711588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.720540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.727585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 143/287] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.729235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.729901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.730178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.730445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] PATH_INFO: `/v3/users/1d832541a1454ccbb27567c9c720f370` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.731586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] RBAC: Authorizing `identity:get_user(user_id=1d832541a1454ccbb27567c9c720f370)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.733204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.734994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 145/288] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/users/1d832541a1454ccbb27567c9c720f370 => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.735474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.749928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.900157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4823bc-e32b-48f9-9148-b44fc404b8f7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:27.922875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:27.957592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:27.958671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:27.959092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:27.959503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] PATH_INFO: `/v3/users/3b76c2f8af864ca0ac89ca60767e8d56` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:27.961247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] RBAC: Authorizing `identity:get_user(user_id=3b76c2f8af864ca0ac89ca60767e8d56)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:27.963872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:27.966912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 144/289] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/users/3b76c2f8af864ca0ac89ca60767e8d56 => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:27.975561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:27.997836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb51b7f-f9e0-4066-9bfb-0a4e0dfbfeba None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.065051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.065713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.065990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.066258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.068070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.069790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.071456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 146/290] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.082382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.107347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.184928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e4c04-3154-4457-9afc-e68fb9a40284 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.258708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815105d-bab9-4e29-90ef-f5e2b34af077 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.273620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.274255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.274537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.274808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.286484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.305732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.306419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.306699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.306972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/users/1d832541a1454ccbb27567c9c720f370/roles/b646c18b962b467d83b73b821972238a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.309778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, user_id=1d832541a1454ccbb27567c9c720f370, role_id=b646c18b962b467d83b73b821972238a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.313047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.336767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 147/291] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:28 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/users/1d832541a1454ccbb27567c9c720f370/roles/b646c18b962b467d83b73b821972238a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:28.394056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.395739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.482404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.531353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 145/292] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:28.538313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.653189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce031c2-4f80-4dff-8524-0a297730b3f4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.689085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.689707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.689995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.690273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.691408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.692861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67841f91-68fc-46f3-9ae7-209d13bfa8a8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.694929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 148/293] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:28.700980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.822045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1bedf0-35a8-4552-9948-1775f667b34d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.871420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:28.872034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.872307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.872571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.873421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=3b76c2f8af864ca0ac89ca60767e8d56) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:28.873723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:28.875181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d7eac-cd9d-4b66-8c35-9384a96f8d22 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:28.881280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 146/294] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/role_assignments?user.id=3b76c2f8af864ca0ac89ca60767e8d56&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.476982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8327-7aec-4acc-b777-5c028ca761dd None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.477579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8327-7aec-4acc-b777-5c028ca761dd None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.477579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef8327-7aec-4acc-b777-5c028ca761dd None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.479110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 149/295] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:29.484816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84cd58-02ef-4e4a-a4b1-9703156ead0a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.485362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84cd58-02ef-4e4a-a4b1-9703156ead0a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.485439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a84cd58-02ef-4e4a-a4b1-9703156ead0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.501813 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdfbb2-3307-4e11-ac60-2989a9d80b51 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.502126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdfbb2-3307-4e11-ac60-2989a9d80b51 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.502362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cdfbb2-3307-4e11-ac60-2989a9d80b51 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.504050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 150/296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:29.508432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6a84cd58-02ef-4e4a-a4b1-9703156ead0a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.509383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6a84cd58-02ef-4e4a-a4b1-9703156ead0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:29.510784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470e43-6285-4855-a199-eb2ace7a37df None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.511280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470e43-6285-4855-a199-eb2ace7a37df None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.512073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9470e43-6285-4855-a199-eb2ace7a37df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.540030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d9470e43-6285-4855-a199-eb2ace7a37df None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.542312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d9470e43-6285-4855-a199-eb2ace7a37df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:29.551960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.617187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.617390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 147/297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:29.639901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 151/298] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:29.896037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6480e91-fb6d-44bd-b336-895a2ecdb620 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.896578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6480e91-fb6d-44bd-b336-895a2ecdb620 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.897047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6480e91-fb6d-44bd-b336-895a2ecdb620 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.899024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 148/299] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:29.904970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.905335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.905748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6480e91-fb6d-44bd-b336-895a2ecdb620 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.905850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.926215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.927716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:30.027668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 152/300] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:30.028935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.032413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6480e91-fb6d-44bd-b336-895a2ecdb620 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.069632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.070081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.070363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.070635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.081426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:30.086580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.087470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:30.088715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 149/301] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:30.093779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.119829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.162896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1716bd-1253-40ed-814f-aaf69f4ea3c8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.181779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.210787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.211960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.212343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.212831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.222368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641e4115-37dc-4ee2-b57a-b0620fc8a9b5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.227192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:30.232977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.233981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:30.234764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:30.240010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 153/302] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:30.241684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.268699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.269808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.270292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.270779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.271621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.272221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.273938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.289274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 150/303] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:30.291441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.374639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71bde6-bbc3-4a61-a688-4c67aeed8227 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.384975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.405720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89743216-54e2-4f01-876b-67de66b14c91 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.412210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.413334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.413995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.414263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.415387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.415926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.417539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.427466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 154/304] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:30.430261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.450770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.451575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.451854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.452482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] PATH_INFO: `/v3/users/3b501cae74c946a28ff6b231f2c68d93` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.453991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] RBAC: Authorizing `identity:get_user(user_id=3b501cae74c946a28ff6b231f2c68d93)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.455971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.460877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 151/305] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/3b501cae74c946a28ff6b231f2c68d93 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:30.461915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.474153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.552878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff4da4b-52e0-42a3-b573-a20fd21a2b4b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.575267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2aa614-81ba-4349-9af6-0812b2bcba08 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.588753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.589424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.589706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.589994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.591139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.592572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.594180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 155/306] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.597029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.615863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.616495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.616782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.617075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] PATH_INFO: `/v3/users/6cba4fbf365a47eb9aa417873d6486d8` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.632112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6cba4fbf365a47eb9aa417873d6486d8)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.634030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.635706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 152/307] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/6cba4fbf365a47eb9aa417873d6486d8 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.636458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.646011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.720219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455af714-d8b5-4a48-965b-a3ba8365b5fa None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.722694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.754735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8687d7fc-f152-4167-b813-59270019e0f9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.767755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.768655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.768985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.769267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] PATH_INFO: `/v3/system/users/3b501cae74c946a28ff6b231f2c68d93/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.771630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3b501cae74c946a28ff6b231f2c68d93, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.773421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.796928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 156/308] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 13 19:36:30 2022] PUT /identity/v3/system/users/3b501cae74c946a28ff6b231f2c68d93/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:30.796928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.809793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.810598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.810967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.811356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] PATH_INFO: `/v3/groups` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.812235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.813926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc53f1c-b5b6-4a7f-a54c-1becbd95c324 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.837079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 153/309] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 13 19:36:30 2022] POST /identity/v3/groups => generated 229 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:30.887450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b119f-3f8b-4950-b0e6-7d75475bbf25 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.887780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b119f-3f8b-4950-b0e6-7d75475bbf25 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.888039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419b119f-3f8b-4950-b0e6-7d75475bbf25 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.889798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 154/310] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:30.895757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.896096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.896366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.916687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:30.918132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:30.939960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afa947-904b-4eda-bdda-d002066416f8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.994335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.995429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.995890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.996656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.998128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.999407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, user.id=6cba4fbf365a47eb9aa417873d6486d8, scope.system=all) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.000075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.003142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.008439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 155/311] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:31.016648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 157/312] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/role_assignments?user.id=6cba4fbf365a47eb9aa417873d6486d8&scope.system=all&role.id=b646c18b962b467d83b73b821972238a => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.272434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.377349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063084a0-263a-4ee6-af66-02755fef61dc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.419518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.420217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.420500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.420769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.432389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:31.437281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:31.438177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:31.438960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 156/313] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:31.445725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.487418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.562819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d4d40-acf3-4d9a-b057-dc87a5881e28 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.598404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.599039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.599314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.599587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.600265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:31.600565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.602004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.611799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 158/314] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.617525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.681634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.722843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45696519-cd3d-4026-85f3-8a3b7298bfbf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.758638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.759304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.759580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.759851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] PATH_INFO: `/v3/users/1d832541a1454ccbb27567c9c720f370` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.761264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] RBAC: Authorizing `identity:get_user(user_id=1d832541a1454ccbb27567c9c720f370)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.762894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.769299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 157/315] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/users/1d832541a1454ccbb27567c9c720f370 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.774179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.881398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.886359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabead93-6ce1-413f-8316-6447e76ffcde None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.922796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.923415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.923690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.923959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.998520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.999983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.093959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.103939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.104391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 159/316] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:32.207716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136ca07f-5efd-4572-a649-e9be40362374 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.231667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.243291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.244189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.244468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.244785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.245960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.247457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20bc18-b71e-426e-9730-210781f7f833 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.249376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 158/317] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.258916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.360468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c70f0e-e910-4160-8324-07a68bcd25fe None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.395528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.396203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.396480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.396751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.397563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, scope.project.id=4b3fef2046454438aa0708862af8a381, user.id=1d832541a1454ccbb27567c9c720f370) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.397871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.399319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319bb6-c9e1-4fe0-943d-ea0333374e4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.405954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 160/318] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/role_assignments?user.id=1d832541a1454ccbb27567c9c720f370&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b646c18b962b467d83b73b821972238a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.095128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244319-ecf3-4b0d-bd3a-268682891598 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.095668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244319-ecf3-4b0d-bd3a-268682891598 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.095668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c244319-ecf3-4b0d-bd3a-268682891598 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.097155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 159/319] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:33.102267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642da86-dd86-4174-813a-b88d8deed57c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.102677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642da86-dd86-4174-813a-b88d8deed57c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.102748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b642da86-dd86-4174-813a-b88d8deed57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.121500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b642da86-dd86-4174-813a-b88d8deed57c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:33.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b642da86-dd86-4174-813a-b88d8deed57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:33.179656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af0de-90f6-4c70-aaa5-17cbacf1a06c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.179907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af0de-90f6-4c70-aaa5-17cbacf1a06c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.180187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2af0de-90f6-4c70-aaa5-17cbacf1a06c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.181682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 160/320] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:33.186687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.186963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.187233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.189666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.204815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 161/321] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:33.210906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:33.212976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:33.258424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb136-4009-4ae7-a4d0-130c30517160 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.258905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb136-4009-4ae7-a4d0-130c30517160 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.259355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebb136-4009-4ae7-a4d0-130c30517160 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.261962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 162/322] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:33.268804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.269295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.269704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.292665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:33.293868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:33.312924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 161/323] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:33.362055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 163/324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:33.466876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.574483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.580475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9da07-0c89-48cb-b57c-6346446c1e99 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.605451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.618416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.619068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.619347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.619613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.630048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:33.634795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:33.635672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 162/325] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:33.637657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.679772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c6eeb-c8a4-4aa3-b6da-d0a59a80d9b2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.698203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.716721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.717433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.717707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.717972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.719084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.721037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.722661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 164/326] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:33.724898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.738702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5f5d82-bfc4-4c83-a6cc-91084410f4e1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.772509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.773168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.773435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.773702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.784772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:33.789620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:33.790567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 163/327] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:33.791511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.827761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab52f7-2e37-434a-807c-3c67263d9a0e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.840105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.862410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.863034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.863305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.863573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.864260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.864584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.866049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.875277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 165/328] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.876128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.902555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ecf54-7d13-43f2-a6c7-ccf636506214 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.925291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.937694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.938323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.938593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.938861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] PATH_INFO: `/v3/groups` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.939490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.940977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.956825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 164/329] 10.222.0.18 () {60 vars in 1134 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:33.957511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.982935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7070ba2f-4e7d-4898-8a9a-33f658b1d0af None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.030463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.031232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.031515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.031765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.032494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.032792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.034291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.043505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 166/330] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.049345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.086022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c0f9be-237c-461e-9164-3b93141e6f5d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.088477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.128373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.129197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.129431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.129627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] PATH_INFO: `/v3/users/3b501cae74c946a28ff6b231f2c68d93` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.130901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] RBAC: Authorizing `identity:get_user(user_id=3b501cae74c946a28ff6b231f2c68d93)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.133637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.134083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 165/331] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/users/3b501cae74c946a28ff6b231f2c68d93 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.140113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.162833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf83c9ac-46e9-4e47-8b77-493eff10d853 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.175155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.203847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.204503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.204782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.205085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] PATH_INFO: `/v3/users/6cba4fbf365a47eb9aa417873d6486d8` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.207171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] RBAC: Authorizing `identity:get_user(user_id=6cba4fbf365a47eb9aa417873d6486d8)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.208616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.211794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 167/332] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/users/6cba4fbf365a47eb9aa417873d6486d8 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:34.215982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.266019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2cae-eb74-4274-839a-62573d5595ff None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.311871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.312853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.313355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.313718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.314454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] RBAC: Adding query filter params (role.id=7e3d833e6ff343faa2a2b56eadd5cc59, user.id=3b501cae74c946a28ff6b231f2c68d93, scope.system=all) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.315341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.316203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df1bf-71b2-43dd-a3db-afeb4b9f747c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.331897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 166/333] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/role_assignments?user.id=3b501cae74c946a28ff6b231f2c68d93&scope.system=all&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 510 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.360137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fddc4-2344-4526-8405-610b8dc2daa5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.400448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.401218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.401496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.401769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] PATH_INFO: `/v3/system/users/6cba4fbf365a47eb9aa417873d6486d8/roles/b646c18b962b467d83b73b821972238a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.403470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6cba4fbf365a47eb9aa417873d6486d8, role_id=b646c18b962b467d83b73b821972238a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.404930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe67fbe-d314-4c23-b746-1b926e8709e7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.416598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 168/334] 10.222.0.18 () {58 vars in 1334 bytes} [Tue Sep 13 19:36:34 2022] PUT /identity/v3/system/users/6cba4fbf365a47eb9aa417873d6486d8/roles/b646c18b962b467d83b73b821972238a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:36.826816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22705-8f39-4355-8799-c4b2e62d1567 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.827349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22705-8f39-4355-8799-c4b2e62d1567 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.827349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f22705-8f39-4355-8799-c4b2e62d1567 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.828720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 167/335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:36.834569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5360-1724-4376-98f3-27c29dbec6e4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.835052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5360-1724-4376-98f3-27c29dbec6e4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.835172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f5360-1724-4376-98f3-27c29dbec6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.861978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a10f5360-1724-4376-98f3-27c29dbec6e4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:36.863186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a10f5360-1724-4376-98f3-27c29dbec6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:36.881029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde3ce-6493-408b-a041-cbc1008ddfb0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.881338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde3ce-6493-408b-a041-cbc1008ddfb0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.881821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcde3ce-6493-408b-a041-cbc1008ddfb0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.883555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 168/336] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:36.890501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8bc2-8ba0-4d51-8090-a3cd5bacdd28 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.890782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8bc2-8ba0-4d51-8090-a3cd5bacdd28 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.891055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f8bc2-8ba0-4d51-8090-a3cd5bacdd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.919029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e54f8bc2-8ba0-4d51-8090-a3cd5bacdd28 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:36.920469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e54f8bc2-8ba0-4d51-8090-a3cd5bacdd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:36.938783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.969409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.991094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 169/337] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:36.992463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7fc8a-607d-4263-9a56-f8ecc540405f None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.992739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7fc8a-607d-4263-9a56-f8ecc540405f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.993045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7fc8a-607d-4263-9a56-f8ecc540405f None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.994507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 170/338] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:37.010379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804abd-1660-466e-a4d4-6ed235a0cb0c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.010710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804abd-1660-466e-a4d4-6ed235a0cb0c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.011442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34804abd-1660-466e-a4d4-6ed235a0cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.054644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-34804abd-1660-466e-a4d4-6ed235a0cb0c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:37.056983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-34804abd-1660-466e-a4d4-6ed235a0cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:37.074680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 169/339] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:37.165103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 171/340] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 19:36:37.294167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e74e-cb0a-4253-9a9d-78c72809a800 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.294812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e74e-cb0a-4253-9a9d-78c72809a800 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.295303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816e74e-cb0a-4253-9a9d-78c72809a800 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.297126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 170/341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:37.310792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.311102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.311329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.339863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:37.341098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:37.353902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816e74e-cb0a-4253-9a9d-78c72809a800 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.463688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 172/342] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:37.529363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.535239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816e74e-cb0a-4253-9a9d-78c72809a800 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.589605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.590259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.590547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.590815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.598675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:37.609468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:37.610413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:37.611243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 171/343] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:37.616446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.653276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.764465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944b28d-3328-4ac4-a54c-7a4ba9b35ef5 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.776504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645033-29b5-4c0a-b2ba-51f68774ba13 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.824948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.825577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.825855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.826127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.826808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.827108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.828524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.829616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.830288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.830560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.830827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.839008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 172/344] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:37.843383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.863546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:37.868398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:37.869366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:37.870229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:37.877802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 173/345] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:37.878485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.937247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.958981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7435ac6-4f73-4f23-a15b-8a0de69d0269 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.001673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.002420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.002699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.002968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.015312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:38.022362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:38.023308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:38.024226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 173/346] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:38.024968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.064853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.076097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c575ff-7795-4671-ad70-fc2d2e91f4a7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.114120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.114786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.115067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.115337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] PATH_INFO: `/v3/users/6cba4fbf365a47eb9aa417873d6486d8` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.116547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] RBAC: Authorizing `identity:get_user(user_id=6cba4fbf365a47eb9aa417873d6486d8)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.118075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.123147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 174/347] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/users/6cba4fbf365a47eb9aa417873d6486d8 => generated 293 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.123147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.183175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129d1844-d12a-4dac-bdd4-1c252f80c3c0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.240669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.241609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.242056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.242472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.243622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.244080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.246533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.256573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 174/348] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.257824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.260819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.282172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea523-8e60-4b40-9b6e-938d8bdfc555 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.338477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.339419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.339833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.340253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.341309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.341619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.343495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.356597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 175/349] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.357496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.381889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9c7471-9fd7-49cc-abdd-520d39599047 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.389302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.428498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.429267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.429610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.431748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.444483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:38.450561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:38.451501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:38.452290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 175/350] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:38.453318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.506741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038ec4b-08c1-4734-88f0-c2422cb3002f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.538827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.559987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.560611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.560958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.561245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.561978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] RBAC: Adding query filter params (role.id=b646c18b962b467d83b73b821972238a, user.id=6cba4fbf365a47eb9aa417873d6486d8, scope.system=all) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:38.562282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.563773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b95ffa1-80b0-479e-9483-ed44e6547b6f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.588512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 176/351] 10.222.0.18 () {56 vars in 1308 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/role_assignments?user.id=6cba4fbf365a47eb9aa417873d6486d8&scope.system=all&role.id=b646c18b962b467d83b73b821972238a => generated 510 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.593201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d4d0-8669-48fc-849e-943ce76564e8 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.593201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d4d0-8669-48fc-849e-943ce76564e8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.593201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58d4d0-8669-48fc-849e-943ce76564e8 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.593201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 177/352] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:38.593201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58d4d0-8669-48fc-849e-943ce76564e8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.608425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24361e43-19b3-4e01-add8-8156f46f53b7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.658499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.659458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.659895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.660298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.721845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.756943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58d4d0-8669-48fc-849e-943ce76564e8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.759965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.761673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.805870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.806847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.807329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.807771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.861395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.907403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.909510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4058e4d9-2e18-4838-90e3-02004de87573 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.912669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 176/353] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:38.912669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.021953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.050840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4aaca3-0e55-4fe1-95d4-0d5ae5383b5e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.058905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 178/354] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:39.058905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.059139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.059226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.078429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:39.079714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:39.102768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.103145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.103415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.103680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.104423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:39.104737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.106154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.115471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 177/355] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.116328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.144642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.174449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 179/356] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:39.175333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.242922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cd49e0-3581-434f-ac96-5c9397c3a5b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.290509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.291770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.292053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.292481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.293710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.295175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa848d9-40c0-49de-ac34-fbd275d32991 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.296835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 178/357] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.300868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5302a-0dcd-4fd1-be23-34189eb0434d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.300868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5302a-0dcd-4fd1-be23-34189eb0434d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.300868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5302a-0dcd-4fd1-be23-34189eb0434d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.300868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 179/358] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:39.300868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5302a-0dcd-4fd1-be23-34189eb0434d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.326004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc91a6f2-efff-4e34-b449-cf221007bdb9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.339782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.381246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.382236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.382502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.382764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.384713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.386888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.389472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 180/359] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.390099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.424847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.440732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5302a-0dcd-4fd1-be23-34189eb0434d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.482386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.483099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.483380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.483651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.510310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d84149-4d32-4184-8a96-36e202e1a751 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.550408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.551062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.551338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.551969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.560583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.562082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89560b7-937d-4042-bab3-14c3202fda71 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.564099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 181/360] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.564954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.565234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.565499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.570180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.571669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.586274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:39.586274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:39.628912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.675771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 182/361] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:39.677379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.690166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.706299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 180/362] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:39.707168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.821146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bccc34-020e-49eb-a3b2-c19cf77ee1b3 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.829047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f70478-bec6-4ff2-bd03-683ec29ac56d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.868440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.869522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.870091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.870564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.871793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.872427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.872704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.872803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.873005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.874654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.876992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 183/363] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.878249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.885589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:39.891572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:39.892519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:39.893381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:39.896751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 181/364] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:39.897025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.929640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.977151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.000811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deb18c1-2175-4c29-9b56-27872f859672 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.050096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.051086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.052226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.052681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.053960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=4b3fef2046454438aa0708862af8a381) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.054460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.056109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361998d9-6cf8-478b-b098-68f58417bc1d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.056787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.066236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 184/365] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.067101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.110694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.111356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.111739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.112011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.113301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.115857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.122220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 182/366] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.123997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.202241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fbf292-45d0-48c7-915f-879fe8cffce3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.207175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.253193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.253840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.254124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.254391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.255167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=99ea55dfca19477d991d0bd252ed5c98) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.255476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.257027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.264352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 185/367] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b62430e4550a43989b6adfc24920894e => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.265270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.267091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f467c5-f171-4c63-bb83-5c2e949bd65f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.294354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.336665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.337637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.338171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.338574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.339406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.339718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.341236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.352172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 183/368] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.357347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.405824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4431b871-044b-4245-ad57-17869ff7f18c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.460709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.461848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.462367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.462842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.478232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:40.483895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:40.485006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:40.485896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 186/369] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:40.486624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.521027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d87d36-05d0-4441-8e59-c2ae6fbcf76d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.523880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.567378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.568307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.568729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.569240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.571724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.573757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.578945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 184/370] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.580101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.628622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73a7b0-04e4-49f3-a667-56c8a3eca123 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.639386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.678512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.679239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.679625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.680080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.703975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49563a-3739-4446-9947-9f4ae9455ccd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.760488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.761316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.761693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.762058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.763146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.763543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.766240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.778903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 185/371] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.779756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.792203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.794111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.832824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.900092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec4360c-fe06-4fea-b249-a04cbea20c84 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.916402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.935264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 187/372] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:40.935883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.948306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.949221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.949597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.949968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.950864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=99ea55dfca19477d991d0bd252ed5c98) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:40.951332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.953676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.962656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 186/373] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.964240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.073931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af956c0-48a1-4b55-991f-be886b82db05 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.109684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.124737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.125407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.125680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.125949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.182597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b476514-d3d2-44d9-9c5a-cf4fd3ce55fa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.263440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.264009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.264288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.264555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.265769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.267271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.269375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 187/374] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.277940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.278130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.279435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.364308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.443354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 188/375] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:41.449620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.450970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182111ba-46a9-4d2f-91c4-b5e4209cda9a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.484101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.496671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.497358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.497629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.497892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.501634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.503139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.504869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 188/376] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:41.511170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.565190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93b53f-1618-4170-b83a-602e9a85bc0c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.602060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.602713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.603086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.603380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] PATH_INFO: `/v3/projects/af00e3aad1ad462b983823ea889f0033` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.604630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] RBAC: Authorizing `identity:get_project(project_id=af00e3aad1ad462b983823ea889f0033)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.606217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.613524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 189/377] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/projects/af00e3aad1ad462b983823ea889f0033 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.615667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.681697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360af2-8607-428c-bff0-c1f49fb3c2a2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.706341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.740599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.741264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.741543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.741818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.742554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=4b3fef2046454438aa0708862af8a381) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:41.742859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.744368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.746632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.756959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e3c28-6599-48e2-b12e-6f90c47daf9f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.768973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 189/378] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b62430e4550a43989b6adfc24920894e => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:41.805820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.806536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.806822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.807092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] PATH_INFO: `/v3/groups/c069038194cc4740b0d5e1b7d68961b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.817865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] RBAC: Authorizing `identity:get_group(group_id=c069038194cc4740b0d5e1b7d68961b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.819298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c360daa-7ff4-444c-8d48-4a599120af89 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.821457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 190/379] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/groups/c069038194cc4740b0d5e1b7d68961b1 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.827928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.960287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338f2ee-e4d5-450b-80aa-95d0e81d0577 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.010429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.011066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.011352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.011622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.012374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] RBAC: Adding query filter params (group.id=c069038194cc4740b0d5e1b7d68961b1, role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=af00e3aad1ad462b983823ea889f0033) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:42.012684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.014198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7303c-718b-48f3-8149-f00876dafd28 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.024336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 190/380] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/role_assignments?group.id=c069038194cc4740b0d5e1b7d68961b1&scope.project.id=af00e3aad1ad462b983823ea889f0033&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:42.847292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89c25-2061-4b82-bfbc-525c28a64ca0 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.847856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89c25-2061-4b82-bfbc-525c28a64ca0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.847856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b89c25-2061-4b82-bfbc-525c28a64ca0 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.849459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 191/381] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:42.855203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.855491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.855767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.879942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:42.881195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:42.932618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.941784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172754b0-1731-40da-baf0-3c8a759a7a1f None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.942096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172754b0-1731-40da-baf0-3c8a759a7a1f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.942321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172754b0-1731-40da-baf0-3c8a759a7a1f None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.943670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 192/382] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:42.949442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.949723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.949994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.987218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:42.993045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 191/383] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:42.995349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:43.082313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 193/384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:43.315438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.440783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9671fcd-54ae-41af-b3a0-208d80db9c22 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.482051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.483015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.483475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.483955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.497360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:43.502821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.503810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:43.504748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:43.507364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 192/385] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:43.513703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.526022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.581982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.669327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab929814-94cb-4838-bf43-5a41d86111e3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.679794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.694772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa56328c-1c0b-4de6-ba2e-8ced7f131300 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.711231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.711860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.712255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.712536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.723345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-02f09989-a8cf-4623-94e2-ebdd0276690d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:43.728288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.729186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:43.730407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 193/386] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:43.731067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbe75-f067-4830-be67-ccabdd8a50a0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.731395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbe75-f067-4830-be67-ccabdd8a50a0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.731488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbe75-f067-4830-be67-ccabdd8a50a0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.732724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 194/387] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:43.738779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dbe75-f067-4830-be67-ccabdd8a50a0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.741346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.742034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.742311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.742579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.743250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] RBAC: Adding query filter params (name=member) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.743613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.745075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b27e-82a4-4fde-8dc1-1684c28a5762 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.754243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 194/388] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.755532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.755812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.756077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.775837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:43.777604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:43.851998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 195/389] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:43.856917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.877319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.908021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dbe75-f067-4830-be67-ccabdd8a50a0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.964730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.965389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.965664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.965929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.966624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.966890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.968287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.993517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 195/390] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:43.999416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.018175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.038678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe944c4-cf26-477d-84ee-1102140a7d2e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.095118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.095775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.096053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.096320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.105926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.107396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.112406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 196/391] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.115945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.235787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f41af-c341-4486-8365-b6fd771134bd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.256403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca9b85-3501-4c8e-89dd-cb057b997874 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.281023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.281799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.282144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.282484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.283837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.285564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.287746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 197/392] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.288635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.304156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.304850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.305318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.305577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.306952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.308634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.310818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.312059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 196/393] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:44.313116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.347531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.402691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f172aa-601f-484c-8149-e3ec5dd4dc44 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.476446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217bba7-adb3-4fa4-b069-2bdecb2ebc6a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.481809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.483115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.483771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.484388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.505502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:44.515661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.516425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.516804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.517155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/groups/0d506bcaa9af41dbb594660e31c0b990/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:44.517972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:44.519777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:44.521538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:44.522805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, group_id=0d506bcaa9af41dbb594660e31c0b990, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.523096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.531313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 198/394] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:44.531313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.547034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 197/395] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 19:36:44 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/groups/0d506bcaa9af41dbb594660e31c0b990/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:44.549133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.622127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.650140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52646f26-c034-47ba-840a-104fc5af7650 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.694235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.694801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.695052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.695326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.697591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.699047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a8ad-2cde-4f60-a4f0-3350555d604c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.700725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 199/396] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.701978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f80b3-0a4c-46c4-89d8-3fcd4d8337ce None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.702254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f80b3-0a4c-46c4-89d8-3fcd4d8337ce None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.702568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f80b3-0a4c-46c4-89d8-3fcd4d8337ce None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.704034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 200/397] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:44.709334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.709720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.710082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.720472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.733393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:44.735117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:44.746933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285283c-aff8-4c40-8443-c099b63ecbfa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.791347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.791963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.792237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.792503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.793217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:44.793515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.794902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.807965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 198/398] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.809050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.827199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 201/399] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:44.829052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.931820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.947416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed95b9-e416-49ce-a7a1-0eabf5767583 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.963108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9acfe8-af1b-4e8b-9e12-469798ac7afb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.990073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.990892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.991297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.991671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/groups/0d506bcaa9af41dbb594660e31c0b990/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.994664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, group_id=0d506bcaa9af41dbb594660e31c0b990, role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.996351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41877be1-dc30-4cf5-8394-a74a8852dc82 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.011798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.012738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.013176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.013618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.013925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 199/400] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 19:36:44 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/groups/0d506bcaa9af41dbb594660e31c0b990/roles/b62430e4550a43989b6adfc24920894e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:45.015336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770dcb6-70cf-4996-9fac-8db834389581 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.015481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.015670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770dcb6-70cf-4996-9fac-8db834389581 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.015885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e770dcb6-70cf-4996-9fac-8db834389581 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.024958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 200/401] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:45.024958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.024958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.024958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.026463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.028768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 202/402] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.043366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.045336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.065771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:45.068065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:45.197737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 201/403] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:45.206440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.209748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99fb802-d6b1-4c57-a7d7-4c9173f5d296 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.249237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.249874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.250147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.250408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.251543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.253152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.255104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 203/404] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.261740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.293583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.406165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caabcbfc-aeae-4f5b-b5c1-3cd7365baaf3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.424560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d663d96-03ce-4520-9131-e30e287c0c7b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.464282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.471544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.472194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.472473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.472745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.488398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:45.489283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.489977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.490248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.490521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98/groups/0d506bcaa9af41dbb594660e31c0b990/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.492446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] RBAC: Authorizing `identity:create_grant(project_id=99ea55dfca19477d991d0bd252ed5c98, group_id=0d506bcaa9af41dbb594660e31c0b990, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.493948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:45.495443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:45.496304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:45.497128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:45.499429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 202/405] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:45.500440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.517847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 204/406] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 19:36:45 2022] PUT /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98/groups/0d506bcaa9af41dbb594660e31c0b990/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:45.518968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.677294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.709937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f666f6-f3eb-4b02-8b8b-8f95eaee9e4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.742179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa628bc-07b1-4259-b73c-fc9b0aad6837 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.763422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.764592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.765079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.765742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.767013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:45.767699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.769279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.782882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 205/407] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.787900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.788544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.788822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.789127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.790542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.802662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:45.807481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:45.808399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:45.809272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:45.810473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 203/408] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:45.817116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.917276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.972854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05538bc-e959-4a95-ab92-1f02929666d7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.035198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f2e2c-b1d7-4732-81f3-d17d6b383415 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.067897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.069076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.069547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.070023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.076944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.076944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.078501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 206/409] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.083886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.089241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.115672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.116297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.116573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.116831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.117536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:46.117986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.119383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.147182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 204/410] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.152761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.270264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e128f8-3644-4442-ad09-29e84d008f9d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.285364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.303011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e832ad-1732-41a9-9e1e-28e6a168fb14 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.309496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.310430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.310889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.311334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] PATH_INFO: `/v3/groups/c069038194cc4740b0d5e1b7d68961b1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.312807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] RBAC: Authorizing `identity:get_group(group_id=c069038194cc4740b0d5e1b7d68961b1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.314552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.316844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 205/411] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/groups/c069038194cc4740b0d5e1b7d68961b1 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.326882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.356823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.361097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.361097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.361097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.366484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.369032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.373278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 207/412] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.379324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.507859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056bd70a-3f97-4743-b416-73bb68c3bae2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.537367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.580640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.581335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.581609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.581883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] PATH_INFO: `/v3/projects/af00e3aad1ad462b983823ea889f0033` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.583466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] RBAC: Authorizing `identity:get_project(project_id=af00e3aad1ad462b983823ea889f0033)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.585192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.587583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 206/413] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/projects/af00e3aad1ad462b983823ea889f0033 => generated 344 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.593122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.607192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8aa7-1d48-41eb-adb0-a9662cada858 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.659455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.660145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.660422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.660694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381/groups/0d506bcaa9af41dbb594660e31c0b990/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.663514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] RBAC: Authorizing `identity:create_grant(project_id=4b3fef2046454438aa0708862af8a381, group_id=0d506bcaa9af41dbb594660e31c0b990, role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.665123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b040ca-a61b-4b30-b4b6-8020621fb74a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.689756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.701316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 208/414] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 19:36:46 2022] PUT /identity/v3/projects/4b3fef2046454438aa0708862af8a381/groups/0d506bcaa9af41dbb594660e31c0b990/roles/b62430e4550a43989b6adfc24920894e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:46.787790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df235b-43bd-4d96-bac8-ae8a35e1039d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.844610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.845344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.845628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.845902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] PATH_INFO: `/v3/projects/af00e3aad1ad462b983823ea889f0033/groups/c069038194cc4740b0d5e1b7d68961b1/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.847924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=af00e3aad1ad462b983823ea889f0033, group_id=c069038194cc4740b0d5e1b7d68961b1, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.849521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cdd8d-94f3-4b43-8845-71844a75b6a7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.881703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 207/415] 10.222.0.18 () {58 vars in 1442 bytes} [Tue Sep 13 19:36:46 2022] PUT /identity/v3/projects/af00e3aad1ad462b983823ea889f0033/groups/c069038194cc4740b0d5e1b7d68961b1/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:46.972527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bc6e-a578-440e-b65e-bc4c992752b2 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.972927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bc6e-a578-440e-b65e-bc4c992752b2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.973460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bc6e-a578-440e-b65e-bc4c992752b2 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.974508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 209/416] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:46.980715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.981053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.981301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.001573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:47.002823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:47.106582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 208/417] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:47.394696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26bc6e-a578-440e-b65e-bc4c992752b2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.528147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26bc6e-a578-440e-b65e-bc4c992752b2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.569718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.577099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.577982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.578450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.579126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.594066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-33985a47-336e-4977-bcf5-9eacc4ffac4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:47.598969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:47.599888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:47.600685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 210/418] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:47.608000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.679472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.770802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e78da-07e0-4fde-b5d5-e10be46c711d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.779150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3cdd6-b88b-472d-87cd-8986d0dcc7c2 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.779448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3cdd6-b88b-472d-87cd-8986d0dcc7c2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.779703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3cdd6-b88b-472d-87cd-8986d0dcc7c2 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.781918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 211/419] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:47.786366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.786906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.787359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.816028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:47.817715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:47.818208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.818850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.819134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.819404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.820084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.820391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.821880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.836770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 209/420] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.847228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.900541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 212/421] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:47.971543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d066f-33ec-4d07-b2d7-ce1b9fc855a0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.973046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.022432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.023057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.023380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.023693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.111962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.113455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.192135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.246703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 210/422] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:48.253014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.273323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.391142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35357ec5-12a7-46cc-952b-169445df0474 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.393340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5846600-e54f-4e6c-93cd-76aba9eca596 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.417268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.433706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.434728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.435194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.435648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.435775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.436364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.436733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.437124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.438262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.440288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a6d3e6-4ada-4462-b8c4-ea21f721dbb8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.446480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 211/423] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.446480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e650-1c23-4087-9d94-68b1751854f2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.446480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e650-1c23-4087-9d94-68b1751854f2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.446480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70e650-1c23-4087-9d94-68b1751854f2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.446480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 212/424] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:48.447664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70e650-1c23-4087-9d94-68b1751854f2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.454295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-5232f292-0507-40cc-a4c8-17dfa94dc281 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:48.459650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:48.460612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:48.461448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:48.464146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 213/425] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:48.465912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.466383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.466835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.486878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:48.488592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:48.535303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.564284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 214/426] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:48.565561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.575907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70e650-1c23-4087-9d94-68b1751854f2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.619517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.620171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.620451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.620722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.621991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.623984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.626329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 213/427] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.634343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.717715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9ddd3-8b5e-4fdc-8ba5-8d61de273c1d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.741891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.793420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.794434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.795065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.796621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.797734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.798616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.800790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.819156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 215/428] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.826411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.855197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31ace1-0703-4f32-862f-9eb9b5a8c1e0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.907828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.908439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.908712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.909010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.909738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=4b3fef2046454438aa0708862af8a381) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.910050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.911955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.920452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 214/429] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.925197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.925792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.965421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1da88c0-d17a-439b-a646-ebb2f4b6ed07 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.045756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.046408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.046688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.046963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.083127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9a838-6ee0-431a-9c8f-ca6fb23db1d9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.134600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.162320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.163001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.164073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.164493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] PATH_INFO: `/v3/roles/member` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.176764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.178317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.187161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:49.197827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:49.198801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:49.199598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 215/430] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:49.205970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.307206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.319879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 216/431] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:49.326041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.370648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29940282-4cd6-4b2a-9bc8-ace59b8ca25a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.407462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.409102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.409589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.410090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.410945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] RBAC: Adding query filter params (name=member) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:49.411452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.413165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.423108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 216/432] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:49.429822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.440538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.474722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a02e7-2a8f-40f7-a001-3612c4d409e2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.511434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.512074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.512362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.512679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.513813 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.515280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d18a0-ede2-4bf1-a44d-4ca4dcba070a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.517060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 217/433] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:49.518337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d615c-6cd7-40db-8787-a3dcbb9a66cd None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.518621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d615c-6cd7-40db-8787-a3dcbb9a66cd None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.518891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d615c-6cd7-40db-8787-a3dcbb9a66cd None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.524691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 218/434] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:49.524691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d615c-6cd7-40db-8787-a3dcbb9a66cd None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.550147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86defa8-2f09-4b3f-9481-241eb8a3af6d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.576198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.590182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.591332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.591628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.591884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.637754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d615c-6cd7-40db-8787-a3dcbb9a66cd None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.666822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.668300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da45b7a-b02c-4621-9897-0a22742f91f1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.675604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.676261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.676932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.676932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.677977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.679444 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb88b69-1670-4a69-aad6-bae83b3eedef None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.685844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 219/435] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:49.685844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.685844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.685844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.702674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:49.703863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:49.706380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.774335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 220/436] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:49.775196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.781381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 217/437] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:49.781381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c403fd-2d43-4ee9-a039-9e2a3bcb62b5 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.781381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c403fd-2d43-4ee9-a039-9e2a3bcb62b5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.781381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c403fd-2d43-4ee9-a039-9e2a3bcb62b5 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.781381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 218/438] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:49.782009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c403fd-2d43-4ee9-a039-9e2a3bcb62b5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.808219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.887995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c403fd-2d43-4ee9-a039-9e2a3bcb62b5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.909541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad275fb-5386-4b2e-a36c-8aec69e67486 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.923246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.924263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.924531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.924795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] PATH_INFO: `/v3/projects/99ea55dfca19477d991d0bd252ed5c98` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.925971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] RBAC: Authorizing `identity:get_project(project_id=99ea55dfca19477d991d0bd252ed5c98)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.927409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00fd7f-fdf5-433a-8b41-dd95713921db None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.929068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 219/439] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/projects/99ea55dfca19477d991d0bd252ed5c98 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:49.930231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.930571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.930844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.949501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:49.950699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:49.951749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.952375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.952652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.952956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.953685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=99ea55dfca19477d991d0bd252ed5c98) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:49.954243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.955451 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.961715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 221/440] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=b62430e4550a43989b6adfc24920894e => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:49.961715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.972676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.030356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 220/441] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:50.040795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.075116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.116940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1edc12-f253-4b1e-a885-61dac985bb9b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.159237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.160162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.160607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.160908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.162152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.163658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.165548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 222/442] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.171984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.186270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9ecb2-5013-4470-9871-e95c4c357df8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.236714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.237771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.238089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.238359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.252768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.253132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:50.258207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:50.259564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:50.260378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 221/443] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:50.267836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.290983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef32ed-c16d-4748-847f-c4a7e03f63e5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.327227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.327913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.328148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.328453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.329309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=7e3d833e6ff343faa2a2b56eadd5cc59, scope.project.id=99ea55dfca19477d991d0bd252ed5c98) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:50.329430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.330917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.337514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 223/444] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=99ea55dfca19477d991d0bd252ed5c98&role.id=7e3d833e6ff343faa2a2b56eadd5cc59 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.338708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.357317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.379339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff05b5-c79b-40f0-888d-9831b75ee867 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.438069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.438934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.439394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.439853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.441048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:50.441555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.444077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.457420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ed00a-4246-4eb8-8056-a1ec5f64c2cd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.458497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 222/445] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.464673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.511340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.512546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.513042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.513510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.517024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.528508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:50.533648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:50.534671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:50.535507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 224/446] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:50.541067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.594109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de419090-35fd-4290-9654-151534cdfdcd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.635046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.635951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.636239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.636519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.655285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.657770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e29e8-14d8-4a8f-9299-1245e1714c18 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.696779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.697673 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.698086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.698643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.699771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:50.700620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.701553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.712943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 225/447] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:50.719183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.721617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.723149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.788559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.834305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759cf914-c02f-4d61-b212-71908b3fdb2d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.839583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 223/448] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:50.844533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.876267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.876781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.877093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.877363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.897684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.950102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.951955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.956994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef22b2-23c3-4869-afd7-8796e31f764e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.000665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.001356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.001634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.001915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] PATH_INFO: `/v3/projects/4b3fef2046454438aa0708862af8a381` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.003070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] RBAC: Authorizing `identity:get_project(project_id=4b3fef2046454438aa0708862af8a381)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.004540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.010764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 224/449] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/projects/4b3fef2046454438aa0708862af8a381 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:51.012045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.019785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.077669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 226/450] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:51.083612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.125339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c756f-f8fa-4c05-b227-f4972fb4d2c4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.138278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.161095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.161726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.161993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.162253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] PATH_INFO: `/v3/groups/0d506bcaa9af41dbb594660e31c0b990` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.163371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] RBAC: Authorizing `identity:get_group(group_id=0d506bcaa9af41dbb594660e31c0b990)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.164860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.166466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 225/451] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/groups/0d506bcaa9af41dbb594660e31c0b990 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.172387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.182187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafcb9f-649c-4e72-a6f2-dfe3427a9182 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.216001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.216547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.216829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.217126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] PATH_INFO: `/v3/projects/af00e3aad1ad462b983823ea889f0033` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.218231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] RBAC: Authorizing `identity:get_project(project_id=af00e3aad1ad462b983823ea889f0033)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.219710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.221376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 227/452] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/projects/af00e3aad1ad462b983823ea889f0033 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.226962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.253532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.276134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dac09d-de5c-460e-b3e8-62d93f76b003 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.310046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.310655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.310933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.311204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.311932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] RBAC: Adding query filter params (group.id=0d506bcaa9af41dbb594660e31c0b990, role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=4b3fef2046454438aa0708862af8a381) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:51.312235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.313718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.332915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 226/453] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/role_assignments?group.id=0d506bcaa9af41dbb594660e31c0b990&scope.project.id=4b3fef2046454438aa0708862af8a381&role.id=b62430e4550a43989b6adfc24920894e => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.334644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93162188-add5-4ec9-8eb8-802176aad2ea None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.371355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.372008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.372281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.372543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] PATH_INFO: `/v3/groups/c069038194cc4740b0d5e1b7d68961b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.373742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] RBAC: Authorizing `identity:get_group(group_id=c069038194cc4740b0d5e1b7d68961b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.375230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43799538-26c2-44e8-8e8c-37d1ba9470a5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.376931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 228/454] 10.222.0.18 () {56 vars in 1180 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/groups/c069038194cc4740b0d5e1b7d68961b1 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:51.383629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.436355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.499065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc363765-c810-4a99-848b-8788b72aed11 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.541728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.542338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.542615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.542884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.543609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] RBAC: Adding query filter params (group.id=c069038194cc4740b0d5e1b7d68961b1, role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=af00e3aad1ad462b983823ea889f0033) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:51.543912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.545412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fc531-633c-4c55-9fcf-66dc7aa46244 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.552001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 227/455] 10.222.0.18 () {56 vars in 1376 bytes} [Tue Sep 13 19:36:51 2022] GET /identity/v3/role_assignments?group.id=c069038194cc4740b0d5e1b7d68961b1&scope.project.id=af00e3aad1ad462b983823ea889f0033&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:54.243358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d9d8-8e70-4387-a9b3-db5cbf56ca77 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.244382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d9d8-8e70-4387-a9b3-db5cbf56ca77 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.244857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d9d8-8e70-4387-a9b3-db5cbf56ca77 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.246734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 229/456] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:54.254144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8dc1-7e8e-43df-8958-b8b0e2febdb7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.255146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8dc1-7e8e-43df-8958-b8b0e2febdb7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.255617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d8dc1-7e8e-43df-8958-b8b0e2febdb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.283781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4a8d8dc1-7e8e-43df-8958-b8b0e2febdb7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.285362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4a8d8dc1-7e8e-43df-8958-b8b0e2febdb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.389029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 228/457] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:54.472572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c9b-87ad-4400-b33c-f29d35b8161a None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.473219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c9b-87ad-4400-b33c-f29d35b8161a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.473688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432c9b-87ad-4400-b33c-f29d35b8161a None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.475244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 230/458] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:54.481147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977841a-76e6-41e5-97b6-365068abcb37 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.481786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977841a-76e6-41e5-97b6-365068abcb37 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.482287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977841a-76e6-41e5-97b6-365068abcb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.504109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b977841a-76e6-41e5-97b6-365068abcb37 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.505679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b977841a-76e6-41e5-97b6-365068abcb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.539011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.588731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 229/459] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:54.702484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532f07-3aa1-4215-9f9f-4b9f6bcf7072 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.702811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532f07-3aa1-4215-9f9f-4b9f6bcf7072 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.703028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f532f07-3aa1-4215-9f9f-4b9f6bcf7072 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.704699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 231/460] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:54.710526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.710815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.711089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.741291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.741896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f532f07-3aa1-4215-9f9f-4b9f6bcf7072 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.742517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.867379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 230/461] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:54.891156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.913765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.937709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f532f07-3aa1-4215-9f9f-4b9f6bcf7072 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.013917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.014580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.014860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.015130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.029803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867d1f7-baea-43da-a004-6077e3d14138 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.048159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.052921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.057997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d6096923-709c-404e-b1bc-7819b250ea31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:55.070890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.071866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:55.073234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:55.073234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.073234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:55.073234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 232/462] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:55.073649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.073649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.073937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.074027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a5a64-f88b-43b7-ad4c-32b3716abbe1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.074181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.074271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a5a64-f88b-43b7-ad4c-32b3716abbe1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.074438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1a5a64-f88b-43b7-ad4c-32b3716abbe1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.080058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 233/463] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:55.080916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a5a64-f88b-43b7-ad4c-32b3716abbe1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.089037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.091404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a60acd26-5660-4c88-8b1d-fcf2fbc55cf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:55.114395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.115434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 231/464] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.117560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.118438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.136665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:55.137884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:55.207634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 232/465] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:55.208467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.208999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1a5a64-f88b-43b7-ad4c-32b3716abbe1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.246842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.248261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.248901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.249373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.250348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.251000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.252689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.267126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 234/466] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.268663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.281306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.291059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.311521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2af3df-4416-4bba-b40c-5c95406d09a9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.346391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.347020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.347296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.347564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.348230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.348515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.349943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.368423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 233/467] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.369287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.389856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2c979-4919-408d-9489-157f95a6a051 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.430318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.431480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.431981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.432520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.444834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.446763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:55.464051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.465052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:55.465906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:55.465906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.465906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:55.468325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 235/468] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:55.469577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.473543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3372c-e3d5-4de4-9b02-45ae9896f010 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.509412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.510616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.510616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.510814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.511396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.512964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.526899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.539108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.548944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 234/469] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/users => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:55.548944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.600466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f17cb-5cb5-4c2a-a120-1fb6c7225be3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.639289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.639890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.640170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.640448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.641144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.642642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.670468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 236/470] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:55.671567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.688252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4f0412-4294-4fab-a5fb-789096145ae0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.768622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a6131-4600-462c-b339-229f0219097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.769173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a6131-4600-462c-b339-229f0219097c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.769305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a6131-4600-462c-b339-229f0219097c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.769553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a6131-4600-462c-b339-229f0219097c None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.770202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a6131-4600-462c-b339-229f0219097c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.770506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a6131-4600-462c-b339-229f0219097c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.772141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a6131-4600-462c-b339-229f0219097c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.785150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 235/471] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.792531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a6131-4600-462c-b339-229f0219097c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.848173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70a523-cd2b-44f7-aa8d-8217b614126d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.867553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.871121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.903340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.903921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.904203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.904527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.918518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.920096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.929549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a6131-4600-462c-b339-229f0219097c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ac74e24e-4353-4c13-8bf0-3f1c15f21fbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:55.937855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.939200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:55.940333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:55.940333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:55.940333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:55.940333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 237/472] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:55.940333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbbf2e-5c61-4a6b-9203-5b2a231da616 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.940681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbbf2e-5c61-4a6b-9203-5b2a231da616 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.940758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbbf2e-5c61-4a6b-9203-5b2a231da616 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.942720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 238/473] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:55.944972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbbf2e-5c61-4a6b-9203-5b2a231da616 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.996128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.997625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.997625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.997625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.998323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.000829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb578e-847c-4074-b45b-0627b4a50749 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.035732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 236/474] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:55 2022] POST /identity/v3/users => generated 248 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:56.037474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.037956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.038424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.065608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:56.067518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:56.093059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fbbf2e-5c61-4a6b-9203-5b2a231da616 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.137004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.137646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.137925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.138196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.139060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:56.139367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.145662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.155992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 239/475] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:56.161358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:56.162199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 237/476] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:56.205323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.315927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd800ba-ccf6-4fde-a552-06aae340d61b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.352593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.353296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.353572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.353841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.354494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.355982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.383053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 240/477] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:56 2022] POST /identity/v3/users => generated 249 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:56.540568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:56.593625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.677920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92a85e-ab1a-420a-b72e-341b3e7784fb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.719249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.719965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.720317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.720593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.721379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.722987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.734043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.sql.core [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:36:56.734043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:36:56.734043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [parameters: {'id': '01f57b8db09b41ad81ebfe50a9bf9f4f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:36:56.734043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93091) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:56.752021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:56.753023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application yield resource Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise self.value Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application session.commit() Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:36:56.753920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:36:56.754799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:36:56.755670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application result.read() Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:36:56.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [parameters: {'id': '01f57b8db09b41ad81ebfe50a9bf9f4f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:56.759093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:36:56.760779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:56.762283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 238/478] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 19:36:56 2022] POST /identity/v3/roles => generated 144 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:56.762348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:56.866198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.887250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426daf7a-5e4a-45f2-a36f-d9a7b1248e64 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.924750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.925501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.925781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.926053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.937251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-cd9b0738-0a55-4a19-80f6-9e49a9295957 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:56.942548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:56.944685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:56.946635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 241/479] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:56.954276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.119985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a88c7-1040-40d8-a427-132bd6322cf9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.132013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.170154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.171093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.171547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.172022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.173274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:57.173753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.175408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21674b0-112c-4ab0-b32f-fb597ea8572f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.185516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 239/480] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.144484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2441a-6c4c-4adf-a67b-14ab024446f7 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.145626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2441a-6c4c-4adf-a67b-14ab024446f7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.146110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2441a-6c4c-4adf-a67b-14ab024446f7 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.148109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 242/481] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:58.153583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bd090-0bea-44b1-81e3-836e52ab7175 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.154553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bd090-0bea-44b1-81e3-836e52ab7175 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.155059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4bd090-0bea-44b1-81e3-836e52ab7175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.181595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1d4bd090-0bea-44b1-81e3-836e52ab7175 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:58.183188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1d4bd090-0bea-44b1-81e3-836e52ab7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:58.262156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 240/482] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:58.577888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac707d4-a904-4df4-97ee-9d64e37ec02d None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.578278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac707d4-a904-4df4-97ee-9d64e37ec02d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.578457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac707d4-a904-4df4-97ee-9d64e37ec02d None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.580042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 243/483] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:58.585506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.585760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.586030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.613866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:58.615056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:58.661734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac707d4-a904-4df4-97ee-9d64e37ec02d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.718270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 241/484] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:58.812064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac707d4-a904-4df4-97ee-9d64e37ec02d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.866779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.867337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.867608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.867870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.880040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.889598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d89eeb25-3330-4498-9d8e-ff92fa8f3dfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:58.894362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:58.895299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:58.896093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 244/485] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:58.901589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.933506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50c58-788c-49dc-aa51-110720e91510 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.933822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50c58-788c-49dc-aa51-110720e91510 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.934050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee50c58-788c-49dc-aa51-110720e91510 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.935635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 245/486] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:58.940615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.951820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.952095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.952359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.978522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:58.979735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:59.057332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e913b1-18ad-447c-a0fa-6f78580685ef None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.077776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 246/487] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:59.079371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.108875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.110696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.112575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.114513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.115492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.116026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.117689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.136644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 242/488] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:59.143840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.275726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec2831-10a3-436a-9277-7fa9a71027b8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.328094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.335157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.345914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c1436-c891-4ae2-8590-383b4ee4c610 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.349261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.350555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.350555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.350555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.364916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-4456b04a-391b-4696-bdec-de3a951cb7db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:59.369888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:59.370780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:36:59.371626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 247/489] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:59.381479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19819-83c4-4078-8dda-32f172d8d7ed None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.381765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19819-83c4-4078-8dda-32f172d8d7ed None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.382034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19819-83c4-4078-8dda-32f172d8d7ed None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.383569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 248/490] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:36:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:59.385691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19819-83c4-4078-8dda-32f172d8d7ed None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.407873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.408517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.408791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.409086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.420841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.422710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-464fc448-2b01-4ee5-9e1b-4a08b711c7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:59.437880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:59.438807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:59.439647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:59.439647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:59.439647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:59.439647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 243/491] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:59.440209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.440414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.440685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.459459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:59.460680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:59.528379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19819-83c4-4078-8dda-32f172d8d7ed None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.532975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 244/492] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:59.532975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.577103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127407aa-7582-4067-b317-0c70656d843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.577837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127407aa-7582-4067-b317-0c70656d843c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.578163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127407aa-7582-4067-b317-0c70656d843c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.578472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127407aa-7582-4067-b317-0c70656d843c None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.579267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127407aa-7582-4067-b317-0c70656d843c None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.579768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127407aa-7582-4067-b317-0c70656d843c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.581533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127407aa-7582-4067-b317-0c70656d843c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.599429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 249/493] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:59.601084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127407aa-7582-4067-b317-0c70656d843c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.618763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.657167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc74034-9c2a-4a2b-aee1-a16aa1c6fbc8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.691464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.706051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.707009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.707711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.708205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.722786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:59.729858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:59.730724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:36:59.731715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:59.734248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 245/494] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:59.735254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.759051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127407aa-7582-4067-b317-0c70656d843c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.801862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.802869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.803280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.803680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.804457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.805006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.806701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.822024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 250/495] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:59.823142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.854762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50170e43-698e-425d-baef-cdb14ec52f71 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.890383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.896671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.898096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.898375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.898668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.910238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.911623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:59.926789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:59.927827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:59.928677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:59.928677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:59.928677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:36:59.928677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 246/496] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:59.929661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.935690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f6775c-6e9a-4440-97f9-70c5d794cbd4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.949830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.971260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.971899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.972180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.972545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.973348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.973656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.975190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.984652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 251/497] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:59.985527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.047805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07fc12-a7da-4c04-bb90-077af1efa890 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.086201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.086993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.087268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.087540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.101877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:00.106092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49876e-5612-49ed-8519-f506b639b503 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:00.108012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.108960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:00.109798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 247/498] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.112225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.151307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.151981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.152259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.152522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] PATH_INFO: `/v3/users/nova` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.161481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.162077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.162976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:00.177648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.178595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:00.179522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 252/499] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.180110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.218379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.226686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed1394-5a88-4517-9e76-b83d0ad68693 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.259605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.260233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.260509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.260788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.261491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.261802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.263220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.275071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 248/500] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.275915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.286912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfcdc8-fa13-4cf6-bc99-f38ad9e6242d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.321505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.322141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.322443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.322733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.333346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.334930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:00.348693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.349852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:00.350755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:00.350755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:00.350755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:00.350755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 253/501] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.350981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.379205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f33bfb8-be34-425b-b0b9-c84df6c8144f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.381577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.415592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.416109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.416373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.416640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.417319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.417615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.418973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f2581-535b-4c3a-9331-7d2ca90b0469 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.427285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 249/502] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.428560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcebb-2105-444f-bf7e-34ab2d085138 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.428843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcebb-2105-444f-bf7e-34ab2d085138 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.429142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffcebb-2105-444f-bf7e-34ab2d085138 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.433291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 250/503] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:00.433291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffcebb-2105-444f-bf7e-34ab2d085138 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.436772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.452169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d892c887-a47b-484b-96b0-fd4a5d4ecf8f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.486186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.486820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.487089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.487363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.488077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.488353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.489918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.528011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 254/504] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.528849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.536481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffcebb-2105-444f-bf7e-34ab2d085138 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.571192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.571826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.572090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.572357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.579388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.580838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:00.593566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.594445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:00.595333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 251/505] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.595878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.622860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.633165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d63811-561a-41c8-ad05-098b47e219e7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.666806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.667337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.667617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.667889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.668528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.668831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.670254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236230f1-01e6-44c9-bbbe-4d4b9dbfeb83 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.682438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 255/506] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.683605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.683938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.684224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.691034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.697336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0e8e4-29e8-4c24-9496-ec5799bcde91 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.701863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:00.703051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:00.731223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.731856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.732132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.732402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.742816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.744315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:00.758894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.759950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:00.760947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:00.760947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:00.760947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:00.764812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 252/507] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.765519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.773276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 256/508] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:00.774162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.850287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.874410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafcbee-409b-4e59-87fa-9962d4787ec0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.878266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef9b0-c79c-44b6-acbe-20363afa10e0 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.909872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.910591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.910869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.911132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.913387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.914119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.914467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.914810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.915605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.915986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.917569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.919292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.941318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 257/509] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.941318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.986730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.988295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.048408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9093dd7-684b-41b4-8d30-61702e6d032b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.072133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.081878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.082555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.082810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.083075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.090301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.091780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.095631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 253/510] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:01.096546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:01.103275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.104338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 258/511] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.105527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.159166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.199046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940eada9-f244-4781-a078-8dc822fbec98 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.209945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1d46c-db53-4a60-8908-7c39c170e0e4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.243043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.243728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.244165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.244571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.245596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:01.246076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.248341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.257085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.257784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.258130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.258510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.262340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 254/512] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.263580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.305574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.336025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.337638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.363879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51379af-6bb9-46de-bc82-ab1c4617ab00 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.391568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.403005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.403652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.403927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.404195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.414607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.416090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:01.432213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.433299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:01.434182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:01.434182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.434182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:01.434182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 255/513] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.435070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.450817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 259/514] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:01.450817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.542231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.553519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d748c-665b-409b-8f5a-73c16544a6a7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.556644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a253c-7862-445a-842d-896f3bd7f497 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.587912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.588677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.589057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.589481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.591992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.592598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.592871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.593168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.593888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:01.594187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.595631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.600680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.602193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.613829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 260/515] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.614637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:01.618654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.619579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:01.620435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:01.620435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.620435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:01.620435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 256/516] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.620662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.626478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.714216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcaab7c-a324-4e05-a310-c9239a0c35e9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.722718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d535d8-47d2-43df-9a41-621d358e6d93 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.749854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.750525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.750803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.751063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.756082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.756713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.757026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.757293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.757955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:01.758248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.758479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.759667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.759937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:01.771848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.772934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 261/517] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.773825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.777112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 257/518] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.777686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.780372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.843576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.874154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaedd4-23cb-40f7-9092-daac534e703d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.875710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a12ade5-d1ba-41e7-b537-dfb2cefa8b3f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.907089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.907697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.907973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.908239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.909313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.909933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.910206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.910471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.922086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.923544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:01.937408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.938426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:01.939277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:01.939277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:01.939277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:01.939277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 262/519] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.939277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.978149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.979526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.981355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.040758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fea0f2-2aa1-461c-9ba3-87e36ed5dc88 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.063983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.075632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.076247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.076527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.076794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.077491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.077789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.079161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.083902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 258/520] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:02.084731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.095889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 263/521] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.095889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.183842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14572232-e19f-47a8-a403-23d007829d7a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.185397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.189256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e6990-4f3b-4878-9be7-a303b0388553 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.218538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.219153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.219428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.219696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.220330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.221797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.223640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.224311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.224585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.224850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.225598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.225931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.227346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.242469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 259/522] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:02 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:02.243217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.257493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 264/523] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.257493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.307003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.360635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f39135-bbec-4e1b-bb07-b5a4bf43cf84 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.371731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077dedd-7854-4fc8-af1d-57d684868034 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.397533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.398154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.398425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.398692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.399341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.399636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.401085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.412914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.413844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.413951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 260/524] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.414692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.414790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.415285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.427143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.428616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.438599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:02.446271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:02.447363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:02.448274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:02.448274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:02.448274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:02.448274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 265/525] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:02.449158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.531977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce7bc3-0e8d-4824-9134-63dca855a51f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.545152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.554784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3a6583-2b51-4383-a00e-ad44b7885dec None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.569936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.570569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.570844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.571110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.578414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.579868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.589194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.589812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.590090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.590359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:02.592320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:02.593180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:02.594052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 261/526] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:02.595186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.666697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.668145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.677515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.700304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e919ac-735f-43c8-8103-2b8696b2efbc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.737842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.738472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.738749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.739018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.745980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.747435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:02.759039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:02.759939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:02.760803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 262/527] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:02.761305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.765602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.775437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 266/528] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:02.776107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.877761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768aa08-ee94-4c29-9b82-5071334de671 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.880564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec7f95-084b-4a7b-8613-9540fc961e9e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.911155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.912010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.912289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.912554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.913222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.913521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.914615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.914882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.915226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.915503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.915773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.916458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.916758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.918238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.923280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.933002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 263/529] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.933694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.942149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 267/530] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.942915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.999684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.039299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d499-83cc-4ebf-b1c9-00b230cab1da None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.053309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da93317-e90a-4a6a-854f-33a563a05b26 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.076097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.076712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.077016 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.077289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.077972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.078275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.079709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.088257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.088911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.089205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.089474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.102604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 264/531] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.102604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.106257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.107801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:03.123698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:03.124693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:03.130474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:03.130474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:03.130474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:03.130474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 268/532] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:03.130474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.175219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.221319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec636bd-b280-4d27-86d4-83e48cc63559 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.241725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd53c2-d46d-4e20-a517-e6b48ac3e320 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.245181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.257354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.258023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.258308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.258572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.265789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.267347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:03.278885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:03.279794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:03.280691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 265/533] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:03.281977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.285809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.286517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.286809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.287084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.287860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.288179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.289857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.305372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 269/534] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.305372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.417884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efdf75-d930-4231-9c5a-b26677d83e59 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.435521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ef010-e11d-4b36-9ea0-b9632438e133 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.440331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.462202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.462872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.463160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.463428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.464272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=2771f4c99b6d4cf3a23a5cdb7af09b7c) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.464588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.466423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.474154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 266/535] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/role_assignments?user.id=2771f4c99b6d4cf3a23a5cdb7af09b7c&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.475144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.494361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.495620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.496142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.496675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.497780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.498347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.508927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.534057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 270/536] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.539647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.581998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.629981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63822cb3-d602-4795-bb18-5dbd37e6d5fb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.673330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.673882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.674448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.674721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.675429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.675734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.677607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.691839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 267/537] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.702997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.736245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfc0bc-8bdb-45a5-94c5-04775b2151ec None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.761500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.821315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b74fbf-925e-4854-92ce-0f90332a5951 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.857732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.858432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.858595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.858769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.858992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.859101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.859353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.859771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.860547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=e306d2cfa2ce47b39520ccaf3daa6a10) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.860868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.862417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.869168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 268/538] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/role_assignments?user.id=e306d2cfa2ce47b39520ccaf3daa6a10&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.880657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.880657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:03.912950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:03.915101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:03.917182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:03.921661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 271/539] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:03.922705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.057604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aceb26-a473-4513-bc71-60de2da945c1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.066432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.099299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.099938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.100212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.100477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.101237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:04.101537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.103064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4634edc-30d4-47a0-a41d-b8bfe632c458 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.117011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 269/540] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:04.123209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.237393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.251778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a90e15-2453-4a66-80cc-bc00bf43d8ed None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.292321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.293347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.293836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.294289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.295213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=6ce3f12892d141c88071095f6fe6506d) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:04.295710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.297755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49f308-552a-4703-b905-c1ab21e4bc6b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.304728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 272/541] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/role_assignments?user.id=6ce3f12892d141c88071095f6fe6506d&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:04.606799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a1ba-3c98-412e-85b8-04e6903a9e17 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.608002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a1ba-3c98-412e-85b8-04e6903a9e17 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.608471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a1ba-3c98-412e-85b8-04e6903a9e17 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.610260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 270/542] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:04.616254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.617158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.617626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.636594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:04.638174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:04.710286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 273/543] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:04.975198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4a1ba-3c98-412e-85b8-04e6903a9e17 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.104732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4a1ba-3c98-412e-85b8-04e6903a9e17 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.151449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.152466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.152977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.153432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.171505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:05.181634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:05.182509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:05.183303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:05.185939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 271/544] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:05.191933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.276686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.294826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398da2f3-2f2a-46d9-9943-81fc325b6941 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.330350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.330979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.331249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.331510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.332177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:05.332480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.334062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.343248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 274/545] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:05.349081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.377805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.469171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80b424-bf9e-46cd-ac9c-b34d8675d3ec None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.519468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.520102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.520534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.520799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.536845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.538385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:05.556919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:05.557937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:05.558984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:05.558984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:05.558984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:05.558984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 272/546] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:05.564989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.676019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ef975-c599-4bbb-a10f-fd0f8055b835 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.711445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.712064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.712335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.712604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.713356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:05.713652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.715045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.727933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 275/547] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:05.733566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.812307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.835407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1f920-51fe-40d2-a556-b3970ceefd3e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.872847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.873505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.873780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.874043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.884365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.885842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:05.899986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:05.902169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:05.905759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 273/548] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:05.907728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.913749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.013724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6245e350-b796-442f-9738-9762dcd62a3f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.060704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.061360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.061629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.061889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.062671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:06.062910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.064370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a2c5df-1a83-4857-8686-7651cc6f2799 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.083366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 276/549] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:06.088984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.121350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9566-6e8c-4e0c-8f88-74b3ea21df39 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.121667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9566-6e8c-4e0c-8f88-74b3ea21df39 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.121870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e9566-6e8c-4e0c-8f88-74b3ea21df39 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.123347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 277/550] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:06.129664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16b036-2bbe-4c09-8e5f-43fe0b057476 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.129949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16b036-2bbe-4c09-8e5f-43fe0b057476 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.130186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea16b036-2bbe-4c09-8e5f-43fe0b057476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.158125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ea16b036-2bbe-4c09-8e5f-43fe0b057476 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:06.159410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ea16b036-2bbe-4c09-8e5f-43fe0b057476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:06.231126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef18d2-78a7-44be-9380-b4b31f040253 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.269857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.270843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.271517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.271958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.282182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.287760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.319391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 278/551] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:06.372410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.374417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.446772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bba24-1cb4-497f-897c-ad99dfe217e8 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.447415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bba24-1cb4-497f-897c-ad99dfe217e8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.447890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bba24-1cb4-497f-897c-ad99dfe217e8 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.453848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 279/552] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:06.458646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.459168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.459662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.492737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 274/553] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:37:06.497344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:06.498299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.499007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:06.595390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.635440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 280/554] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:06.698025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.706579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8231b-6e0e-4ac1-a606-fdafa896927b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.765813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.767270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.767740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.768267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.779444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:06.781226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:06.796954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:06.797928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:06.798803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:06.798803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:06.798803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:06.800928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 275/555] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:06.806106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.851775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dbd46-dda8-45bb-b4fe-7aa11801281a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:06.900143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.917810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:06.919141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.919635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.920149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.933186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:06.938710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:06.939605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:06.940447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:06.943916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 281/556] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:06.949947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:06.970601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:06.984094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289899c-b35f-433c-a776-fa90d9489746 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.019271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.019873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.020324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.020589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.024189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.024189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.024189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.044742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 276/557] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:07.044742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.078685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd13d80b-6222-4ef5-bc98-6b19ac71bd92 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.114168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.114698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.114979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.115233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.115889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.116199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.117604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84439287-dff9-46be-8efb-27c342e2b51d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.126401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 282/558] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:07.127660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd2ec-5abb-43e5-a346-87f9e9a64a7d None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.127940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd2ec-5abb-43e5-a346-87f9e9a64a7d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.128414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8cd2ec-5abb-43e5-a346-87f9e9a64a7d None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.129684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 283/559] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:07.134656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd2ec-5abb-43e5-a346-87f9e9a64a7d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.155887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05dcd7-627f-4fdf-8c9c-70832695912f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.190808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.198721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.199410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.199631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.199885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.210714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:07.215457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.216386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 277/560] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.218001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.232455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.256561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8cd2ec-5abb-43e5-a346-87f9e9a64a7d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.290193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.290823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.291091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.291353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.298249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.299687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-406da9f2-97a7-4caa-882c-ef85d043c679 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:07.310782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.311837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 284/561] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb02ac1-f3b2-4e02-bd20-7f490f5a95bb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.317942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb02ac1-f3b2-4e02-bd20-7f490f5a95bb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.318664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb02ac1-f3b2-4e02-bd20-7f490f5a95bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.320285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679ac03-37a3-40d1-850b-51560e6069b8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.333927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-fbb02ac1-f3b2-4e02-bd20-7f490f5a95bb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:07.335114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fbb02ac1-f3b2-4e02-bd20-7f490f5a95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:07.355558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.356578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.356854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.357154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.367782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.369364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:07.383817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.385036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:07.386070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:07.386070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.386070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:07.386070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 278/562] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.386070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.404925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 285/563] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:07.406217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16369-af72-4a13-845c-11d87f3a1db1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.406496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16369-af72-4a13-845c-11d87f3a1db1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.407190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16369-af72-4a13-845c-11d87f3a1db1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.408958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 286/564] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:07.409493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16369-af72-4a13-845c-11d87f3a1db1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.447761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.458920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.500701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2b329-a4cb-4047-837d-21bf45aa6fff None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.526262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16369-af72-4a13-845c-11d87f3a1db1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.535817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.536426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.536713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.537015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.537682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.537983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.539695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.555442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 279/565] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.555442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.572709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.573355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.573631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.573893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.574614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.574910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.576344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d7f71-4645-4b49-bb72-ebdb78ebf53c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.588646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 287/566] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.594758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.595049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.595298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.614448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:07.615683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:07.662503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4e240-9558-43b2-9768-925c9cb9b7b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.688989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.695182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 288/567] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:07.696049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.700795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.701514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.701863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.702173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.703010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.703371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.705286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.720933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 280/568] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.721139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.732467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.823133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79de95-0862-4cd8-9305-378d1277825f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.838402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d004764-eb56-42aa-a955-984712f49c9b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.863493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.864188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.864467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.864745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.875923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.877385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.877676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.877995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.878273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.878541 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.879294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=5ae58835c32e4a8f9e06f0a4c65ce26b) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.879625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.881081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.887073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 281/569] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/role_assignments?user.id=5ae58835c32e4a8f9e06f0a4c65ce26b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.887959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:07.917554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.918624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:07.919552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:07.919552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:07.919552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:07.919552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 289/570] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.919552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.971611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.019136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.039454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fb05f-4b7a-44dc-b116-a675445cf342 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.049467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb49cac-2c61-414a-952b-5127a6417a75 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.076446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.077120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.077398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.077671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.088763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:08.092682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.093356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.093622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.093890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] PATH_INFO: `/v3/users/nova` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:08.094060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.095089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:08.096073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:08.100266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 282/571] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.101287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.102226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.103664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:08.117727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.118930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 290/572] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.120140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.219059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3eacb3-e528-4984-9062-c614f9a5382f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.231237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.232232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ade76-3d19-44b1-860f-8d10d3731e3c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.260388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.261000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.261280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.261551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.262218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.262540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.263942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.271854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.272711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.273162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.273608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.276637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.277266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 283/573] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.278667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.291761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:08.297364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.298290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:08.299111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 291/574] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.299804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.402750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec715-3edc-4f1b-86b4-b237c40d4851 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.406565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3d93-db05-43c3-ba33-94e4ca66575a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.437855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.438468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.438748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.439016 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.439675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.439975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.441390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.445253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.446053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.447006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.447493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.448625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.449096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.450071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 284/575] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.450852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.450961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.476253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 292/576] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.476253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.494010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.533665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.572565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853c8a9-7d26-4141-abeb-2a3cf8bb0fca None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.584730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ac2e8-e1da-4d38-8c60-a3618c3c1f76 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.609695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.610328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.610607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.610860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] PATH_INFO: `/v3/users/glance` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.619262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.620345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.620706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.621002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.621283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.621554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.622224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.622694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.624597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:08.632763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.633851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 285/577] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.640216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 293/578] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.640216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.731128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.740357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7be403-e8fc-49fc-9141-fe34d60c84db None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.747084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a076bd0-1fc1-4a9a-a30e-e685543a4a94 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.764310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.774854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.775497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.775770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.776030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.786464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.787130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.787410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.787683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.788735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.790276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.800454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.801969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:08.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.806875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:08.807813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:08.807813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.807813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:08.811648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 286/579] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.812498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:08.816952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.817882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:08.823578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:08.823578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.823578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:08.823578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 294/580] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.823578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.921920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905231e-de93-4b40-b611-17158d2a3886 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.941926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa776e-945d-4e4e-a6b8-521b03ec9ed5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.962331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.963064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.963399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.963629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.974947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.976393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.978556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.980024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.980681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.981032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.981489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.982034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.982381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.983924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:08.993591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.994548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:08.995453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:08.995453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:08.995453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:08.995453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 287/581] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:08.997107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.012342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 295/582] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.012342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.044578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.115190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd73a28b-a3f1-4a8e-bc4c-c994397bba88 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.143761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc11b07-d0f3-4c49-9588-86c46e6c1dc0 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.151436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.152079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.152352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.152660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.153355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.153655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.155135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.167288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 288/583] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.168131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.198058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.199381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.199836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.200268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.201296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.201854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.203985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.218104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 296/584] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.219807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.230388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.279836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8807607-7180-438f-91b1-bc02178d3ab6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.315299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.320300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.321024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.321312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.321582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.322365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.322668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.324056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.331577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbcde4-0533-4e92-9fdf-a1f4da1dca25 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.341014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 289/585] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.341014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.371085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.371718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.371985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.372252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.383955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.385563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:09.401423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.402370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:09.408015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:09.408015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:09.408015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:09.408015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 297/586] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.408015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.463289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dacb286-b560-41ad-b708-2131e2956ec3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.479718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.500758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.501423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.501699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.501962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.510759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.512229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:09.523900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.524851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:09.525824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:09.529910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 290/587] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.530624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.533864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa03fef-9019-4b4c-9954-7aea670c249f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.569977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.570923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.571365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.571821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.580035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.581709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.583932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:09.596004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.597314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:09.598785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 298/588] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.603393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.633005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b00c8-619f-43c5-ad31-c46fed5b1f40 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.671178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.671803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.672081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.672350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.680366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.681856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:09.693363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.694386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:09.695356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:09.697273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 291/589] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.698132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.718937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ba897-e3ae-45e5-b854-666c6cbe462c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.734095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.754696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.755405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.755677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.755894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.756621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.756945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.758420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.772173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 299/590] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.773051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.811465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f3d2a-f3e2-4795-88b3-200ce8107a77 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.838721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.847817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.848527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.848869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.849246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.850148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.850535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.852401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.886993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527df7-6b00-4349-8c32-b0a29e1e2d4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.891835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 292/591] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.892401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.930427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.931987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.932267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.932539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.933298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.933603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.935107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.955263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 300/592] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.955263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.010230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.025392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceaa03-893a-49ec-b825-40856eb135dd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.065274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.065849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.066080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.066345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.067154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.067538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.069035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.094436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 293/593] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.095059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.108831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cf6e22-7fa7-4bc2-93d4-50ec797e69db None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.127078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.145272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.145954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.146200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.146468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.153744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.155415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:10.169368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.170999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:10.171963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 301/594] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.172442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.209861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cb76-37ad-4a63-a513-ce240c322b85 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.245932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.246567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.246834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.247099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.257308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.258774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:10.273728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.274958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:10.275956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:10.275956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.275956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:10.275956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 294/595] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.276290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.279190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.294388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa280b63-2425-404a-891c-f1fa5797c60d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.342829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.343758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.344210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.344816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.373045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=844040a2fb60437a9188ccb5293d404b, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.375326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128ef288-e3f1-4dc5-9183-ff71f925d568 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.389526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 302/596] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:10 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:10.391020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa0a6-9064-48a8-b7e1-29f51c75d2f2 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.391304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa0a6-9064-48a8-b7e1-29f51c75d2f2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.391589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fa0a6-9064-48a8-b7e1-29f51c75d2f2 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.393009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 303/597] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:10.393752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fa0a6-9064-48a8-b7e1-29f51c75d2f2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.420580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee85ad3-71f4-43d6-9fd4-7e646a7cb4fb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.436631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.461988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.462633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.462906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.463198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.475526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.477136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:10.492837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.493890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:10.494930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:10.494930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:10.494930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:10.494930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 295/598] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.494930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.552635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fa0a6-9064-48a8-b7e1-29f51c75d2f2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.571968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.605483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.606589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.607180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.607634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.609048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.610032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.612304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a06bca-91e0-4a8b-8657-15fe14d8df2f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.628876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd28a-6f94-4169-83ea-f255b3c5687c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.630923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 304/599] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.636160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.636160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.636160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.658525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:10.659704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:10.681121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.681802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.682081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.682354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.683010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.683312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.684928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.697475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 296/600] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.698328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.761317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 305/601] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:10.761317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.777555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.832547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165004a-f1c4-4f0f-86c6-f5a27991c1d9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.851324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.875756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.876364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.876639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.877033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.877717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.878014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.878502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747d2706-8d23-48e6-8bfb-4287642cc0ff None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.879619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.893467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 297/602] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.894410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.918034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.918990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.919301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.919665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/2771f4c99b6d4cf3a23a5cdb7af09b7c/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.937137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=2771f4c99b6d4cf3a23a5cdb7af09b7c, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.938792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.954315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 306/603] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:10 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/2771f4c99b6d4cf3a23a5cdb7af09b7c/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:10.955517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.017496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c47d9a-cbfb-455b-a2d0-4b4e9c0fb68e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.047317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.066621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.067477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.067846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.068215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.077941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.079847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.092677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67b038-751b-45a5-b721-a3dcde28e0eb None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:11.095274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.096328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:11.097447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:11.100223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 298/604] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.108952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.150452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.165176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.166058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.166474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.166829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.179300 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.182580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:11.196274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.197437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:11.198626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 307/605] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.199203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.240398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80212f-e7d5-45e9-9b10-0b845732fb01 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.286729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.287415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.287802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.288085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.288784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.290581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.304028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 299/606] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.304909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.328175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f1daa-23e3-4357-87aa-bcb0b996c31e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.360081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.372413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.373112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.373632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.373920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.389146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:11.397234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.399283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 308/607] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.401412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.427832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.436709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e071a8-e9e5-4488-9d44-880dd1103a3d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.474951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.475608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.475880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.476193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.476878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.477199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.478619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.490867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 300/608] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.491805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.533785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cedb61-cd4b-43f5-98ba-fb6cab7afa2d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.580356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.581467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.581745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.582078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/e306d2cfa2ce47b39520ccaf3daa6a10/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.595731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88e65e-3cfd-4cb0-b1ed-b572538859a5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.601110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=e306d2cfa2ce47b39520ccaf3daa6a10, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.602560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.621566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 309/609] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:11 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/e306d2cfa2ce47b39520ccaf3daa6a10/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:11.622757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.649997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.650884 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.651157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.651479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.652117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.652419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.655027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.676136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 301/610] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.683249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.711334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.715258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.806666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549baa93-a594-4e71-9f3a-513c70082214 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.841445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0830f88-04c2-45d7-97b6-9e8435ecef4e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.847133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.847855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.848876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.849173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.861241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.862898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:11.882311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:11.883353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:11.885305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:11.885305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:11.885305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:11.885305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 302/611] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:11.890572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.903117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.903708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.903985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.904258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/6ce3f12892d141c88071095f6fe6506d/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.926304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=6ce3f12892d141c88071095f6fe6506d, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.927820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.945413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 310/612] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:11 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/6ce3f12892d141c88071095f6fe6506d/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:12.010360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.074847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9741-9863-478d-adc0-745928e186d3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.148078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.148789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.149108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.149414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.150040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.150337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.151939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.176239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 303/613] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:12.182243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.299093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.317554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60867986-0173-4e43-9478-90c6f2c7bbde None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.377794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.378458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.378894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.379166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.388483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.390002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:12.420571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:12.421809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:12.422751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 311/614] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:12.434512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.696007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce5e11-2d0b-4c10-ab56-8a785d46e0e5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.758176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.774163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.774790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.775067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.775338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.776078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.776380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.777968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.808843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 304/615] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:12.814030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.954654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885cbb90-24eb-405b-9d3c-fb3f51c6cb80 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.999860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.999860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.999860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.999860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.013186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.015119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-a6fa51bb-4043-4305-8091-4bb4dd8f9368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:13.032645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:13.033699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:13.034581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:13.034581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:13.034581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:13.036791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 312/616] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:13.038629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.042937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.121489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6aad6-9cd4-4972-a08a-e12076ca2ebb None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.122477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6aad6-9cd4-4972-a08a-e12076ca2ebb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.122946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6aad6-9cd4-4972-a08a-e12076ca2ebb None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.132541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 313/617] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:13 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:13.134299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25f26-1420-412c-81db-11f80603930c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.134783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25f26-1420-412c-81db-11f80603930c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.135245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a25f26-1420-412c-81db-11f80603930c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.154499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-97a25f26-1420-412c-81db-11f80603930c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:13.156040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-97a25f26-1420-412c-81db-11f80603930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:13.227892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 314/618] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:13.257967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3443ce-4c33-4dff-ad39-4f51832a3f3e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.314635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.315272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.315546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.315800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.316646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:13.316964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.318385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.337511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 305/619] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:13.343099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a25f26-1420-412c-81db-11f80603930c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.498493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a25f26-1420-412c-81db-11f80603930c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.540152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.547763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.548431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.548705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.549007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.557525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.558995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:13.571622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:13.572591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:13.573524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 315/620] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:13.578604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.638660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.644745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.754267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b63e1-2a96-47fa-b0e6-9b1d1a2c272f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.829063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.831298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.832576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.833089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.834566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:13.835224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.837353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1cb8fa-432b-46e2-86f2-9322ce794d2b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.839175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.856656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 306/621] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:13.863787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.889629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.890139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.890413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.890682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.901847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-a0ac0482-f674-4543-88e7-206c9c9b22e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:13.906829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:13.907723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:13.908555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:13.913061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 316/622] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:13.914530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6889d5-8f2a-4fb1-81e9-be465c51bd45 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.914712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6889d5-8f2a-4fb1-81e9-be465c51bd45 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.914935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6889d5-8f2a-4fb1-81e9-be465c51bd45 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.917098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 317/623] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:13.920816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6889d5-8f2a-4fb1-81e9-be465c51bd45 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.976707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.978912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eea91a-3464-4481-9ad2-f0e8b7a76f25 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.987487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.016516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.017639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.018117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.018569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/5ae58835c32e4a8f9e06f0a4c65ce26b/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.035101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=5ae58835c32e4a8f9e06f0a4c65ce26b, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.036862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4e2e4-b7b6-41c8-9d00-a195f6e8c931 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.043713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6889d5-8f2a-4fb1-81e9-be465c51bd45 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.054036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 307/624] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:13 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/5ae58835c32e4a8f9e06f0a4c65ce26b/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:14.055872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4cce-294d-47ae-94d7-f7ceaf10caf2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.056381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4cce-294d-47ae-94d7-f7ceaf10caf2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.057048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e4cce-294d-47ae-94d7-f7ceaf10caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.079745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-224e4cce-294d-47ae-94d7-f7ceaf10caf2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:14.082122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-224e4cce-294d-47ae-94d7-f7ceaf10caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:14.082631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.083258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.083530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.083794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.084461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:14.084756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.086187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.095105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 318/625] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:14.101293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.169379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 308/626] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:14.259060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d8413-6ca1-4d87-911d-2db11d91cc7e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.265030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993ced-0a29-4584-bfc5-a210e68dbfa5 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.265590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993ced-0a29-4584-bfc5-a210e68dbfa5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.265590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9993ced-0a29-4584-bfc5-a210e68dbfa5 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.267367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 309/627] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:14.273169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.273476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.273727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.280729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.297022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:14.298223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:14.305892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.306549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.306829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.307098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.320134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.321619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:14.338401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:14.339415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:14.340293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:14.340293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:14.340293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:14.340293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 319/628] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:14.345895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.387100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 310/629] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:14.458820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6176ffe2-ac17-4ae8-a6d8-8418f83edfa8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.494117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.494743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.495075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.495353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.495995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:14.496298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.497701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.510480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 320/630] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:14.518296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.518445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.547853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.550912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.629916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0934e82-d366-4f95-a90d-391354d402d3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.666317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.667023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.667305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.667585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.676907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.678386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:14.692715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:14.693751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:14.694703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 321/631] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:14.702828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.740089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69fa08b-aa31-4515-a0d4-4609c360e99f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.783576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.784144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.784420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.784752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.800627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:14.805766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:14.806697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 311/632] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:14.808166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.836520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.856727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed8bda-9161-4517-b239-b5a0e2f7c225 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.896713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.900715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.901372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.901653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.901927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.902703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:14.903008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.904459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.923723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f407ac0-6315-4ffa-977a-a9ba9e4fc7c3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.932942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 322/633] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:14.932942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.977020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.977814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.978100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.978371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.993031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:14.999810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.000787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:15.001700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:15.005228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 312/634] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:15.006847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.065591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5360-196a-476d-acb5-79999fdd5321 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.106222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.106831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.107109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.107379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.108046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.108350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.109772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.120942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 323/635] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.120942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.125348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.130900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6c34d-d94e-4545-9829-5447f1896cd8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.167864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.168495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.169095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.169095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.174398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.232632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fbdebe-0db4-42f2-ae3c-db1dc7eaa1cd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.242870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.244339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.268852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.269989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.270466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.270929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.271839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.272356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.273907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159ab78-8f9e-49eb-98df-cc022aad525c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.282417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 324/636] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.283715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd21b-1f48-4dd2-9ee2-486d377574bf None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.284005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd21b-1f48-4dd2-9ee2-486d377574bf None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.284276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acd21b-1f48-4dd2-9ee2-486d377574bf None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.289595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 325/637] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:15.289595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd21b-1f48-4dd2-9ee2-486d377574bf None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.353251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 313/638] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:15.354796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.364739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.389829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.406018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acd21b-1f48-4dd2-9ee2-486d377574bf None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.442278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.443144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.443565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.444004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.455999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbabd8d-3642-4956-abd3-9f266bf4a8ca None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.456548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.458186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d7b4eeac-7cf6-4817-8255-9422459700f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:15.472854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.474067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:15.475188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:15.475188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.475188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:15.475188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 326/639] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:15.475457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.475736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.476000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.495639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:15.497118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:15.501332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.501964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.502242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.502511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.513193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.514658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:15.530365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.531366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:15.532246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:15.532246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.532246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:15.536342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 314/640] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:15.537143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.578423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 327/641] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:15.579442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.641769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.644194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.657961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2203f0-9217-428b-b2c0-8ddbb702ab74 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.703337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.704525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.704525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.704730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.718072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.719593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.721453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c667e6-6195-444f-af40-3234b7e213ac None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:15.736759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.737932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:15.738853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:15.738853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:15.738853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:15.738853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 315/642] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:15.739253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.773856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.774868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.775327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.776002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.777364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.778113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.779700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.802760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 328/643] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.804047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.884067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f1696-7a3f-4029-bf22-9dacf6c2e9aa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.921001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34512a15-e5f0-4801-a830-21cd82533454 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.925394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.926411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.926880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.927407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.928259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.928762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.930414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.931959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.942369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.949060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 316/644] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.950031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.968696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.969754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.970228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.970718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.971773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.972255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.973983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.989084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 329/645] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.990201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.067892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edad12e-66d1-4d67-8c4c-4a483dd8ece4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.102639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0179a98-3a10-459e-8184-d4577609fd77 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.105580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.106141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.106420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.106689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.115259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.117044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:16.129901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.130877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 317/646] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.132463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.143226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.143840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.144125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.144395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.155762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:16.160591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.161574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:16.162428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 330/647] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.163136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.195073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.203890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.250587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a589945-3065-4b69-8d43-ee413a33cfc4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.278872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1635257-739f-49ad-b54a-4bb55e8244ae None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.287700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.288343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.288637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.288904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.297825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.299317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:16.312276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.313676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 318/648] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.314661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.330785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.331582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.331859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.332127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.339996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.341508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:16.353545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.354522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:16.355413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 331/649] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.355906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.426667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0429a-1228-4b46-9e35-a69f2e9a5d0b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.446129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.453238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.461456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.462068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.462348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.462618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.463343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.463644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.465113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.466005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af6784-08b1-4f12-9887-a993c4a03033 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.482587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 319/650] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.483411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.501616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.502237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.502506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.502772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.503553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.503856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.505270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.513169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 332/651] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.513942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.582792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d3cd7-f33b-45aa-b1cf-63bbbe2615b7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.615059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba053545-8ca0-4e54-8297-eccc58e784d5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.615971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.616576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.616849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.617147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.617871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.618166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.619600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd364d-9eb7-4e58-b5bf-d3123af89af9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.636310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 320/652] 10.222.0.18 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.637613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f549-15a0-4031-bf13-286a8b6d7c04 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.637833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f549-15a0-4031-bf13-286a8b6d7c04 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.638101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6f549-15a0-4031-bf13-286a8b6d7c04 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.639282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 321/653] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:16.639969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6f549-15a0-4031-bf13-286a8b6d7c04 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.652817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.653468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.653740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.654003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.654719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.655033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.658423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.671132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 333/654] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.672052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.678632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.684505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.748370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6f549-15a0-4031-bf13-286a8b6d7c04 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.784579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.785245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.785519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.785810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.786922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb6ebf7-a17b-46dd-b0df-8b0a19466ba0 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.822439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.823075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.823351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.823621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.834319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.835780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:16.850760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.851935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:16.852826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:16.852826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:16.852826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:16.852826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 334/655] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.853093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.857674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.859141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5824c44-10fd-48da-a557-9b2f9190aac6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.893991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.916117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.955467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec479a-8062-4306-95e6-6ed93e338a4c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.963538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 322/656] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:16.965363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.965824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.966251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.990953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.991583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.991851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.992133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.993563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:16.995630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:17.063008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.064454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.069235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 323/657] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:17.070087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.127572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.154981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.170554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 335/658] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:17.175888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.193645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f3cf88-3b78-469d-9a25-47af37e49851 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.232253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.232762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.233067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.233337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.234064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.234363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.235843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.242029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 324/659] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.242828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.303461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff42f3-81c5-415a-a108-b70fcfd801b8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.350114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.350763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.351037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.351305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.352006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.352308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.353818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.368039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 336/660] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.368753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.389592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9368a-db6b-44a1-8167-61b3d36219fb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.399629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.429310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.438073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.439095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.439342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.439750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.453726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.455217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:17.471668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:17.473684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:17.473684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:17.473684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:17.473684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 325/661] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.473684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.491998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44fc38b-f912-4e9d-8908-d35778d675de None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.538284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.539189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.539641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.540061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.551971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.553470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:17.568654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.569740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:17.571171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:17.571171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:17.571171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:17.571171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 337/662] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.571171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.600031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9963-3a82-4340-8c9b-fbafff0e9a7d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.636909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.637557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.637833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.638105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.648648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:17.654033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.655077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 326/663] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.662008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.662754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.687029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436cd1a-6e3e-4019-bdb6-25e281080fbf None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.705931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.725894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.726544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.727064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.727064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.736797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.738315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:17.752859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.754104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 338/664] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.755050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.784181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d186a3-9dcb-4c03-9752-b51f3dd245e2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.830553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.831341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.831775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.832055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.832864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.833337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.835309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.851184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 327/665] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.852727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.880709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687506-29ce-42a1-9f3e-0cb52dc01ae8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.936329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.937293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.937770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.938278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.939467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.939923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.942417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.944970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.965856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 339/666] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.965856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.973812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab3de0-f208-45f7-bf85-e4ab27133f1d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.977413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.014718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.015177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.015431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.015747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.016362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.016664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.018083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.035814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 328/667] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.037176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.089033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f09efd-b341-43db-970b-91b4242ca060 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.138344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.139073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.139323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.139605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.140453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.141201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.143903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.171211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 340/668] 10.222.0.18 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.171211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.193327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a540403-e29a-4be1-ab65-791e7da9c596 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.237129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.237959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.239948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.240251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.240523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.250091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.251942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:18.266272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:18.267543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:18.268499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 329/669] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:18.269817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.313341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.315627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2a1f02-ddc5-4e82-a3bf-5021e98e3e0b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.355919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.356569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.356853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.357195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.365422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.366942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:18.379767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:18.380935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 341/670] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:18.381921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.414789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051af8-db79-41ef-8a53-8e9eafed3fb8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.454665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.456432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.456858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.457323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.469501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.471709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:18.490360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:18.491573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:18.492519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:18.492519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:18.492519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:18.492519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 330/671] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:18.493100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.507245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5e924-62e8-4773-86e8-0cdf22cc8b0f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.522360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.550780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.551435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.551732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.552054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.602451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.631404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b48c2d-b0b5-432f-8c3b-9d7a0bb84842 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.639913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.642538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.671009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.671620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.671888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.672172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.672842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.673174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.674603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.688557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 331/672] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.689453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.767595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 342/673] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:18.768557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.788632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.811446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7438f9-4a78-4daa-b8eb-9d52b467095e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.850240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.850857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.851140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.851413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.852112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.852418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.853891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.856539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.873981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 332/674] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.873981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.893018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d88f8-09a0-4d83-9d9b-68e9cb705a6d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.932763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.933416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.933685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.933946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.934590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.934891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.936265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.953522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 343/675] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.953522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.989343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f256e-0447-494c-b44f-e97461f79f5b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.026907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.027893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.028392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.028828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.029727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=2771f4c99b6d4cf3a23a5cdb7af09b7c) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.030200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.031871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.039617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 333/676] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/role_assignments?user.id=2771f4c99b6d4cf3a23a5cdb7af09b7c&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.040746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.052190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.104412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94aabd6-10f0-467e-a88a-e199aa10e4a1 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.121574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.167335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.168758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.169447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.169728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.179152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adc3d0-020f-4d6b-b271-3f7242ed38b5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.187200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.189473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:19.212248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:19.213314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:19.214927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:19.214927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:19.214927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:19.214927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 344/677] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:19.216487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.220862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.221539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.221886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.222157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.222914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=e306d2cfa2ce47b39520ccaf3daa6a10) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.223220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.224719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.244925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 334/678] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/role_assignments?user.id=e306d2cfa2ce47b39520ccaf3daa6a10&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.244925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.368494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564cfde-812d-4858-b281-00b7cbff780e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.404018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.404411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b62b7-a977-4bc0-ae3e-bb5d67e20229 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.420863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.422240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.422881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.423492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.424755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.425485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.427767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.430951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.452918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 335/679] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.453295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.453295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.453295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.453295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.458716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.470138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.471750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:19.486614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:19.487660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:19.488619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 345/680] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:19.493882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.615436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bae98f-6d60-46c3-802d-b71dd3da2eba None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.628695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91c145-daa5-43ac-ab51-d77263038022 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.653414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.654420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.654922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.655396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.663214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.665003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3793c354-011a-43ec-b01a-8cee0dea025d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:19.677498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:19.678506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:19.679407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:19.681517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 336/681] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:19.682957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca94f-2dee-4156-8739-2eb5211484ca None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.683465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca94f-2dee-4156-8739-2eb5211484ca None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.683932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bca94f-2dee-4156-8739-2eb5211484ca None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.685880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 337/682] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:19.687597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.688433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.688791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.689207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.690386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.690796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.691355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.691904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.692368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.692702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.717261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:19.718977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:19.720667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 346/683] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.721785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.765358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.778519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.796636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 338/684] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:19.798102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.902948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efdcf45-ebeb-4248-9535-a044ad4f0bf8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.945005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1a412-05b7-4682-a1c9-6d7a95125d96 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.983362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.983979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.984254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.984518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.008218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.008950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.009233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.009524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.010263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.011585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.013446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.052309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 347/685] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.057376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.098356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.139687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.141770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.215184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da4fd7-cf73-4e7c-a930-74a58529de5d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.217080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.262623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.263255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.263525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.263794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.264519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=6ce3f12892d141c88071095f6fe6506d) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.264819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.266996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.270392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 339/686] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:20.271741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.276964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 348/687] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/role_assignments?user.id=6ce3f12892d141c88071095f6fe6506d&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.278308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.458448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.521277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa5926-0331-47bb-b893-821d5b38ffef None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.539722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2c602-12e8-41fb-8213-31ae90283a6c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.581579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.582780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.583324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.583899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.598106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:20.606753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:20.607889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:20.608818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:20.611572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 340/688] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:20.618745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.620359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.621515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.621959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.622420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.644161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.646547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:20.674184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:20.675535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:20.676652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:20.676652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:20.676652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:20.676652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 349/689] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:20.682772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.726303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.849361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3a1486-1008-4f81-9e5b-b94732b9c780 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.856015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50451f43-7121-4c8c-992e-6f131716f744 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.914292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.915416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.915978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.916531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.917780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.918388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.921366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.935827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.936461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.936737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.937036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.937712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.938077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.939468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.941094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.945458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 350/690] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:20.955967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.957243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 341/691] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.962930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.057271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.091430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ef2bb-b1d6-4764-8caf-97878949d5e4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.140383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.141123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.141530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.141957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.151412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.154561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:21.173536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:21.174886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:21.176356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 351/692] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:21.181663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.218805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf423d5-6b3f-4b93-a97f-dd704c517a0a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.314711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.315390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.315672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.315945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.333503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deccb1c-ad0c-43e6-9f3e-d1070b3e3964 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.345863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.347582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:21.375147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:21.376155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:21.377132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:21.377132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:21.377132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:21.381934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 342/693] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 419 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:21.382933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.383786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.384260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.384735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.385672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:21.386163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.387630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.389924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.408121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.420115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 352/694] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:21.428162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.489155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.608173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640836a1-5890-4599-9b3a-b75fe792308d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.610224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d27c6fc-60d5-41eb-8cc5-7d07a88ad824 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.651177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.651779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.652084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.652361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.653037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:21.653382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.654834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.657097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.658360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.658942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.659470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.660641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=5ae58835c32e4a8f9e06f0a4c65ce26b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:21.661223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.663145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.668118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 343/695] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:21.674164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.680511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 353/696] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/role_assignments?user.id=5ae58835c32e4a8f9e06f0a4c65ce26b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:21.789602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b161950-4445-48c3-b03d-0d7db0aff075 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.804234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.843579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.844745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.845435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.846099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.863178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.865855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-7cdd4c69-5981-414b-b882-2eba5374a843 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:21.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:21.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:21.896666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 344/697] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:21.901061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.995261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.022073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f746-cffd-44bb-8749-0ac09498e417 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.022417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f746-cffd-44bb-8749-0ac09498e417 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.022626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c32f746-cffd-44bb-8749-0ac09498e417 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.026191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 345/698] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:22.030620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e32b-7abf-4e52-808d-5a2375240f5f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.030951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e32b-7abf-4e52-808d-5a2375240f5f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.031161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3e32b-7abf-4e52-808d-5a2375240f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.058604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8529d9-2fe4-484d-b3aa-032294347efa None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.061680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-84c3e32b-7abf-4e52-808d-5a2375240f5f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:22.063281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-84c3e32b-7abf-4e52-808d-5a2375240f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:22.099346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.099985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.100266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.100520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.101281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:22.101587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.103026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.122621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 354/699] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:22.129126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.162837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 346/700] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:22.163982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef239-d96b-4f26-874f-c5bdfa2541a3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.164857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef239-d96b-4f26-874f-c5bdfa2541a3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.165310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eef239-d96b-4f26-874f-c5bdfa2541a3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.167470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 347/701] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:22.181393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.181393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.181393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.203400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:22.204765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:22.265482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc440567-afac-475b-9516-eba853b81a96 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.287014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 348/702] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:22.291667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.307741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.308431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.308714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.309066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.382176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.383766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.490776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 355/703] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:22.496046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.546182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.558277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.605283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.665635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41e06d-c764-4880-a4d3-15c1bf877f62 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.707299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fad84-a61e-4493-bf7c-f11c267e518d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.711220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.711865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.712140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.712412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.726789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.728798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.742615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.743259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.743527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.743791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:22.748569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:22.749554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:22.750831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:22.750831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:22.750831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:22.750831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 349/704] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:22.751457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.763406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.765779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:22.784250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:22.786025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:22.787959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:22.787959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:22.787959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:22.792922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 356/705] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:22.792922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.849635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.874456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c11ae5-e5a5-4832-b8cc-f9638a796774 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.883800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:22.914851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.915739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.916159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.916605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.917364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d525ce-5288-4ae6-8231-8b6255d96a09 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:22.917519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.919474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:22.931695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:22.933013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:22.933916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:22.941088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 350/706] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:22.942297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:22.952454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:22.953107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:22.953383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:22.953648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:22.954286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:22.954581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:22.955980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:22.973000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 357/707] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:22.976601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.065932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd9fa67-9b17-4ff7-8cb0-6f39731d30cf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.093267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449fd634-3af6-4bfd-b052-ec551a3c889e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.110145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.110873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.111244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.112151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.112975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.113730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.115428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.131765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 351/708] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.133665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.140171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.141334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.149477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.150511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.151040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.151581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.153185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.153703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.156069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.167085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 358/709] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.167085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.251125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6e7770-64f7-4c53-8e73-aa0c0eeb5168 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.274575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8729347-a1ce-4eca-8da2-4166acb74b4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.295477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.296341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.296764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.297230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.305072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.306511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.316730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.317447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.317792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.318131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:23.318226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:23.319035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:23.319913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:23.320335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.320335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f889dd-7fe8-4164-b9bb-ac148e1035e4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.324097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 352/710] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:23.324954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.342637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 359/711] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:23 2022] POST /identity/v3/users => generated 254 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:23.343954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb722-1396-43dd-9b7e-86326787df64 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.344303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb722-1396-43dd-9b7e-86326787df64 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.344677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb722-1396-43dd-9b7e-86326787df64 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.346027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 360/712] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:23.349145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb722-1396-43dd-9b7e-86326787df64 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.413616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.427966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.473022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92c484-d038-4388-8b56-92f5a6df2c31 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.486392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb722-1396-43dd-9b7e-86326787df64 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.520146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.521105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.521502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.521875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.523007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.523717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.525465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6e9cc-67b7-43ec-b636-e0e85183bef6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.532322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.533228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.533805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.534439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.535449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.535970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.537752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.539084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 353/713] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.540815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.541167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.541439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.553930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 361/714] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.560555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:23.579670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:23.581342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:23.683078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 354/715] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:23.704578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45645c1a-2a89-4a11-9650-416a03292afe None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:23.720084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:23.743009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:23.743637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:23.743912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:23.744179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:23.744935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:23.745246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:23.746717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:23.753459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 362/716] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:23.982460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.053342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:24.097474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f7f19-5fc8-48c1-94db-b65517ac5ac0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:24.144011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:24.144652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.144951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.145225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] PATH_INFO: `/v3/services/network` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.145806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:24.147254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:24.156347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:24.157409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:24.158244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 355/717] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:23 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:24.163694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.296593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b58de-474a-445f-a2cf-cdec89ca0fc3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:24.331013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:24.331636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.331911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.332181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.332818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] RBAC: Adding query filter params (name=network) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:24.333158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:24.334523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47c754-2893-4a0c-94f9-0805d64f2a96 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:24.339501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 363/718] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:24.345283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.463126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03394b8e-3660-4741-9340-e59da0014b8e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:24.487300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:24.512072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:24.512693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.513007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.513263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.513892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] RBAC: Adding query filter params (type=network) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:24.514184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:24.515553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106bde01-bdcb-461f-8d1e-d9614caea20d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:24.520642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 356/719] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:24 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:24.613473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713add-b72e-46f6-9f44-ef8c48828039 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.614124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713add-b72e-46f6-9f44-ef8c48828039 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.614124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba713add-b72e-46f6-9f44-ef8c48828039 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.615551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 364/720] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:24.621642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.621929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.622208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.643079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:24.644328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:24.736749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 357/721] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:25.041697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba713add-b72e-46f6-9f44-ef8c48828039 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.104931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.181439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba713add-b72e-46f6-9f44-ef8c48828039 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.214429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.215065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.215344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.215611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.228171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:25.232943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:25.233891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:25.234793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:25.237012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 365/722] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:25.240456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.368840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9067b2-231a-4645-b12d-2ef9b7f9cbbe None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.390003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.415582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.416246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.416548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.416804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.417575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:25.417859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.419711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.434062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 358/723] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.440672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.619258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b10c-b57c-4d2b-b66d-840a1c17e599 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.627647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.682495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.683147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.683431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.683703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.694841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.696300 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:25.710543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:25.711622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:25.712920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:25.712920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.712920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:25.712920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 366/724] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:25.719139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.898241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e1d01-b77d-444c-9418-f2ebbe685360 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.953164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.953794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.954071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.954340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.954995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:25.955295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.956722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9621ba0-e955-4e58-ae06-039397ada6f7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.969783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 359/725] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.976242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.150523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e9921-ed3b-41de-9b1a-19d9841af327 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.211058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.211712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.211990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.212874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.218518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b34d6-1c45-4b70-a608-22f2b30c1b96 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.218795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b34d6-1c45-4b70-a608-22f2b30c1b96 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.219076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608b34d6-1c45-4b70-a608-22f2b30c1b96 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.220715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 360/726] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:26.223347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.224814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.228051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.228051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.228051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:26.242760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:26.243774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:26.244726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:26.249144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 367/727] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:26.251951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.275204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:26.276464 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:26.285889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.343252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.422581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 361/728] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:26.443918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a9d44-5de6-4a70-b3c9-7923b1e9f5a7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.502218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.503225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.503694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.504170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.505169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:26.505668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.507341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b79f9-1a80-4963-80af-b552fa669e86 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.535929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 368/729] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:26.541728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.581047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062a45-b3e8-4e00-88f8-a2f1cac86feb None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.581931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062a45-b3e8-4e00-88f8-a2f1cac86feb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.582211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062a45-b3e8-4e00-88f8-a2f1cac86feb None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.584187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 369/730] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:26.585421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b407-4e09-42a7-abfb-e0b87f799b81 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.585704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b407-4e09-42a7-abfb-e0b87f799b81 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.585960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047b407-4e09-42a7-abfb-e0b87f799b81 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.587490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 370/731] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:26.589876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddf89-677b-4709-9764-b33de9ba7a19 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.590222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddf89-677b-4709-9764-b33de9ba7a19 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.590494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ddf89-677b-4709-9764-b33de9ba7a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.611204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-fc5ddf89-677b-4709-9764-b33de9ba7a19 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:26.612529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fc5ddf89-677b-4709-9764-b33de9ba7a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:26.688869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.695432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d74e11-4c20-43ab-b913-94a50dd4c999 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.700630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 371/732] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:26.702570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.703070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.703602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.728208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:26.729810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:26.764753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.765330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.765610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.765881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.801357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.820912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 372/733] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:26.820912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.864228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.865994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.977491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 362/734] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:26.981393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.989770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b9a9e-caa4-4f2f-b4d0-7c2db4b278b3 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.032174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.039381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.040143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.040425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.040693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.053627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:27.059532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.060519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:27.061600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:27.063769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.063928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 373/735] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.063928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.109592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e50cf-df2c-4819-bab9-1cdd2ee8bdca None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.146765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.147381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.147658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.147935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.148603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.150018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.171666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 363/736] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/services => generated 251 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:27.173143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.195719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf23e3-f565-4a3a-9df8-3c01bfd19265 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.267054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.267636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.268041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.268317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.282767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.284674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.302328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a160bf2-041e-4b27-b79f-26f0faf8321c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:27.304194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.305255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:27.306155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:27.306155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:27.306155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:27.306155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 374/737] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.306827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.344544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.355882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.357489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.357811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.358092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.358873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:27.359289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.360773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.363022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.370892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 364/738] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:27.371789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.442430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d399b6-820e-4620-8607-bfbbbe201a54 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.478534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.479180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.479456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.479723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.490429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:27.495214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ff07a-ed2e-4175-b391-b5262248a27f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:27.496086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.497052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 375/739] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.498563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.538648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.539181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.539458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.539727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.540374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:27.540674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.542131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b972fc-126b-4dfd-a546-de1f472979ee None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.565570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 365/740] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:27.567680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa30f-fc66-4667-88d7-df6c4c6639a3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.568084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa30f-fc66-4667-88d7-df6c4c6639a3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.568442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1aa30f-fc66-4667-88d7-df6c4c6639a3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.571106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 366/741] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:27.571903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa30f-fc66-4667-88d7-df6c4c6639a3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.609504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.613978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36baebd-fcb1-41ad-8577-604730716703 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.652574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.653266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.653544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.653816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.666437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.667924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:27.684356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.685462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:27.686379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:27.686379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:27.686379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:27.686379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 376/742] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.687006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.755532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.802636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407eabe3-720a-4224-b753-4133bb6ddad9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.806091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1aa30f-fc66-4667-88d7-df6c4c6639a3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.837607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.838343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.838622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.838878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.838996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.839483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.839758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.840020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.840684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:27.841010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.842386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac8c5f-7151-4df3-babb-f4d38c342374 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.846422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.848037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.855490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 367/743] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:27.855490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.855490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.855490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:27.862362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.863253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:27.864138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:27.868470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 377/744] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.869151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.871138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:27.872329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:27.882301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.953398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 368/745] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:27.954082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.988215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f407f2-85c1-40db-8964-0512eb1a4e0c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.004289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.038704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.039634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.040106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.040567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.041434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:28.041931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.044384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.060013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 378/746] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.061369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.073132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340f1a7-79bb-4232-9413-116c90e0a152 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.110773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.111839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.112332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.112950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.124098 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.126026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.137582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:28.143156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:28.144177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:28.145538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:28.145538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:28.145538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:28.153186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 369/747] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:28.154636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.171443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528fd565-61c8-46e4-966f-2c12d33cba44 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.207909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.208909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.209182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.209460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.210128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:28.210425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.212148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.234015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 379/748] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.234871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.265914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.277665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7654013-a206-4416-94aa-0346abdda7e9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.329011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.329770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.330296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.330595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.341766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.343577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:28.357762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:28.358745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:28.359698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:28.366443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 370/749] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:28.366443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.379862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c200b5-1a26-45e9-8545-d639c8f684d9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.409175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.416079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.416694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.417014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.417387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.418132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:28.418807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.420171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.437310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 380/750] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.437310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.473356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1858be-b422-4d0f-90d6-c9e3a58b01c0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.508845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.508845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.509208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.509480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.510083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.511465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.529059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 371/751] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:28 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:28.529059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.550220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.558274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0353207b-7926-407b-b10e-33cc1134ccb3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.608792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.610438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.610967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.611485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.612440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=5ae58835c32e4a8f9e06f0a4c65ce26b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:28.612979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.614680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.622733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 381/752] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/role_assignments?user.id=5ae58835c32e4a8f9e06f0a4c65ce26b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.623940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.654658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe693d73-bc18-4333-875b-9428e24f2a4b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.688683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.697083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.698081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.698240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.698594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.699410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:28.699780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.701476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.734886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 372/753] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.746247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.750772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4721aa-ef6d-4589-ab3a-aca3656af94c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.794097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.795138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.795631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.795902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.805699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:28.807186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:28.819359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:28.820368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:28.821340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 382/754] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:28.829898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.869350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.905940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8ef38-c6e4-4255-9545-ec70b0175e57 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.967711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.968337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.968609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.968870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.998998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a02b-42ff-4d3c-882e-c79c9e25c3cd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.026303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.077342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.077961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.078233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.078498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.079228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.079533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.084811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.118258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.119783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.123563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 383/755] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.131294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.262574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405aece-a21f-4aa0-a4c7-738b88873782 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.282433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.323809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.324463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.325085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.325355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.344126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.346366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.354458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 373/756] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:29.360103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:29.382023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.382994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:29.383904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:29.383904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.383904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:29.390282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 384/757] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.398059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.423753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.552535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f77c9-d709-48a5-96da-6e93bb825144 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.553512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d82431-aac4-40b9-9be5-3832fdd6dbaf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.596380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.597988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.597988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.597988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.601506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.602288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.602702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.603105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.603876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.604331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.606407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93079dc-772b-4bcf-8ea6-44223aee1c94 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.609690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.611589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.625780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 385/758] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:29.627496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.628598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:29.629743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:29.629743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.629743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:29.630146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e425-5f25-4c65-9a4c-8db63f9b6c42 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.630598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e425-5f25-4c65-9a4c-8db63f9b6c42 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.631014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3e425-5f25-4c65-9a4c-8db63f9b6c42 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.631540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 374/759] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:29.633152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 386/760] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:29.637718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.638771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e425-5f25-4c65-9a4c-8db63f9b6c42 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.671181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.741105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.763685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32356fcd-31af-4df5-84c6-fb7858cb3449 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.765566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3e425-5f25-4c65-9a4c-8db63f9b6c42 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.801994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.802805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.803148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.803490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.805590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.806486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.807375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.807835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.808647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.809140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.810668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33406e5-2dcd-4237-b413-2d22d73246ff None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.813468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.814855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.824364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 387/761] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.825847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.826312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.826747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:29.831709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.832579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:29.833508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 375/762] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.834081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.849094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:29.850569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:29.937297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 388/763] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:29.937918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.938042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.979105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dba46-2ad6-4386-81cd-9c54618d54d5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.067984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.068637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.068922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.070011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.083707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.085205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.088397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:30.110522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.111618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:30.112544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 376/764] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.120505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.143941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96564e95-b58f-49fc-8d64-f318bd1b22c5 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.217770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.218357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.218637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.218908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.219603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.219909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.222733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.246753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 389/765] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.255836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.296477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d8c1f-b740-43c6-9cd2-1cec5b7af8ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.333942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.334591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.335585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.335585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.335901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.336208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.337683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.351870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 377/766] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.360633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.368199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4cc6a-d3eb-425a-bfda-06750ee63acd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.368574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.402569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.412382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.413448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.413919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.414799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.417001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=844040a2fb60437a9188ccb5293d404b, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.419032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.436254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 390/767] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:30 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:30.437548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.528243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf2efe-a1e6-4e08-a7f4-825edd832aba None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.554979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d75b-5eea-47b0-876d-8ae491eb30c4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.569961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.570603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.570938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.571144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.571779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.573388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.583372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.584440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:30.589787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 378/768] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.593148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.621378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.622065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.622342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.622653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.623454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=be2777d5f726425e9d2c144b5d98f715) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.623797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.629855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.644471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 391/769] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/role_assignments?user.id=be2777d5f726425e9d2c144b5d98f715&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.697358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.724319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d32cf-badf-4fbe-bdcf-73274e26d0c7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.778814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.779817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.780332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.780802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.781737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.782326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.784170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.790684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 379/770] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:30.797026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.867385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.919232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8601a2-c09a-47b1-8671-f6cf0a6de540 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.955655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.956293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.956554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.956825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.957481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.957791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.959157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1d742-14eb-4882-b962-4a24ac4950f0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.965680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 392/771] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.971700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.977578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cecef-b02d-4605-9fc4-ee96f9b7fc1e None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.977905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cecef-b02d-4605-9fc4-ee96f9b7fc1e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.978127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cecef-b02d-4605-9fc4-ee96f9b7fc1e None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.981024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 393/772] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:30.990510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9caefb-42e8-423d-a6f3-c889b0ec0a32 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.990863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9caefb-42e8-423d-a6f3-c889b0ec0a32 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.991129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9caefb-42e8-423d-a6f3-c889b0ec0a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.013726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3c9caefb-42e8-423d-a6f3-c889b0ec0a32 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:31.015213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3c9caefb-42e8-423d-a6f3-c889b0ec0a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:31.089593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 394/773] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:31.103054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65b22-7402-40fa-8e71-40c6e397aeb1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.103419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65b22-7402-40fa-8e71-40c6e397aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.103604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea65b22-7402-40fa-8e71-40c6e397aeb1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.105236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 395/774] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:31.110323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.110607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.110880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.147497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177058-b8c7-49e1-bfc0-ed519a9f9e97 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.152012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:31.155490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:31.209863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.216502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.217258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.217547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.217819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] RBAC: Adding query filter params (interface=public, service_id=ff6458b25ef74450bf07a996e77b8676, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.218849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.220282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.241852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.245960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 380/775] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/endpoints?service_id=ff6458b25ef74450bf07a996e77b8676&interface=public®ion_id=RegionOne => generated 194 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.251322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.269004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 396/776] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:31.379770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.425282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2012da-9908-4304-b1b0-6126e7819254 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.501072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.501699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.502038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.502308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.502918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.504303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.515993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 381/777] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/services => generated 772 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:31.522337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef476a8-bedd-4a34-8dd9-a6f56a185cee None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.570616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.571299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.571569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.571839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.586226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:31.591107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.592066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:31.592944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 397/778] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:31.598865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.650172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.696763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.726543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.783702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e383ef-fa53-4421-91f7-bae287cc8ab3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.832519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.833289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.833552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.833818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.834568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.834907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.836311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.841679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b212a6-1150-48bb-9648-094fbbaa1481 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.846548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 382/779] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.854872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.895624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.896382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.896667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.896959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] PATH_INFO: `/v3/services/network` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.897657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.899092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.911722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.912753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:31.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:31.914385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 398/780] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/services/network => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.921408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.976115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24253df-079a-4df5-9d88-102e885504d6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.001071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.014677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a6376-608a-41e7-9122-31c91720148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.015573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a6376-608a-41e7-9122-31c91720148f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.015848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a6376-608a-41e7-9122-31c91720148f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.016115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a6376-608a-41e7-9122-31c91720148f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.032315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a6376-608a-41e7-9122-31c91720148f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.034313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a6376-608a-41e7-9122-31c91720148f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-838a6376-608a-41e7-9122-31c91720148f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:32.051783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.053028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:32.054037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:32.054037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:32.054037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:32.054037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 383/781] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:32.058850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a6376-608a-41e7-9122-31c91720148f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.066284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7525b0e-a645-4963-b590-31720d0a2a65 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.068220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.116284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.117267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.117795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.118263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.119073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] RBAC: Adding query filter params (name=network) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.119590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.121173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.131657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 399/782] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:32.138260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.251854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a6376-608a-41e7-9122-31c91720148f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.275070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62154d52-9f1f-434d-bce0-f4dd6a51acba None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.319672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.320308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.320588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.320860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.322063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] RBAC: Adding query filter params (type=network) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.322372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.323754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.325233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.325941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.326221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.326484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.327127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.327425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.328789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.339307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 400/783] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.345166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.347177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 384/784] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:32.352248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.396597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.429266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.503118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5caf-843b-4d75-9f5e-892343c34598 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.539058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.539806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.540145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.540477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.541269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] RBAC: Adding query filter params (interface=public, service_id=2b5630b450c04ba7be6cc1ffce143b60, region_id=RegionOne) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.541635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.543403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.543547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f951f6e9-7dea-45ca-a43b-dcc05eab08cd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.551809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 401/785] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/endpoints?service_id=2b5630b450c04ba7be6cc1ffce143b60&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:32.557354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.594598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.595456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.595840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.596215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.607573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.610492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:32.625306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.626302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:32.627476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 385/786] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:32.637221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.773027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.785128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3b9bb5-e210-4ec1-aae0-e5668a551a1e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.849168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.853014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.853686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.853932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.854226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.854815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.856449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.863747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ed77c-1362-4c61-a81f-f08f01a8d7fa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.870197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 402/787] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/services => generated 772 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:32.909557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.910372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.910737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.911084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.912088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.912490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.914628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.948374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 386/788] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.954056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.244292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304d27a-565f-4628-923d-8933f95f376d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.291625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.292229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.292510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.292783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.306254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.308055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-98ac7c6b-9a0c-4eb1-bbf6-35277f1b1ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:33.325211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:33.326337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:33.327904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:33.327904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:33.327904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:33.327904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 403/789] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:33.332630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.411291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe9bf-b007-490f-a537-3c023af2419d None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.411907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe9bf-b007-490f-a537-3c023af2419d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.412279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bfe9bf-b007-490f-a537-3c023af2419d None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.414166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 404/790] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:33.420180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de06d1-8a90-42cc-8f77-57b564565045 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.423350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de06d1-8a90-42cc-8f77-57b564565045 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.423871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de06d1-8a90-42cc-8f77-57b564565045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.445218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-54de06d1-8a90-42cc-8f77-57b564565045 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:33.446769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-54de06d1-8a90-42cc-8f77-57b564565045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:33.450869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.469190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ad605-a6e5-4403-897b-d336eacbbf85 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.509083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.509786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.510078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.510349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.510995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:33.511295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.512804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac70ff8-f003-4298-be70-f3e0aae89f60 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.524163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 405/791] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:33.525515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f63f9-0e1a-4ba9-9b7b-8aca5d340307 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.525803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f63f9-0e1a-4ba9-9b7b-8aca5d340307 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.526168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f63f9-0e1a-4ba9-9b7b-8aca5d340307 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.527779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 406/792] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:33.530307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 387/793] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.536174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f63f9-0e1a-4ba9-9b7b-8aca5d340307 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.539560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.540011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.540454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.582363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:33.585720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:33.597522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.694001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 388/794] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:33.732366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f63f9-0e1a-4ba9-9b7b-8aca5d340307 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.770000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.770696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.770968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.771235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.778863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.780323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.783287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:33.793711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:33.794693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:33.795579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 407/795] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:33.802600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.821081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.985266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50cd879-1906-4ddf-a162-d94cc2089a14 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.001707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54398cca-0e57-45d6-8bde-2c27cc9d3bd7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.057117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.058202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.058484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.058754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.065138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.066193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.066663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.067133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.068024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.068522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.070286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.071104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:34.076228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:34.078012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 408/796] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:34.079500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.096682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 389/797] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:34.098809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.104854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.167244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.234494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899e8cf-5df3-486b-9fd4-a3bbd0a2db9c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.255595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d0888-e87e-4287-a699-e4ffb77df6ad None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.281395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.282037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.282342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.282567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.294513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:34.299763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:34.300772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:34.301642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:34.305924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 409/798] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:34.306966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.310116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.312318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.312836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.313424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.314415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.314916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.316599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.329513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 390/799] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:34.330689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.399192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.439343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba7151-5937-4eb6-adae-bce87c89cd5d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.445346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301f628-81b9-410a-9589-ef4c7a68e193 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.466211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.484522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.485230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.485501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.485681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.485823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/5ae58835c32e4a8f9e06f0a4c65ce26b/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.486300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.486586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.486858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.487532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.487839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.488393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=5ae58835c32e4a8f9e06f0a4c65ce26b, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.489262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058ec3-3121-42d1-a6f6-fe08c12328f5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.490147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.500965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 391/800] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:34.502975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a32a1-0cdf-45bb-a7fa-08b12663022c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.503500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a32a1-0cdf-45bb-a7fa-08b12663022c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.503808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a32a1-0cdf-45bb-a7fa-08b12663022c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.509003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 392/801] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:34.509003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a32a1-0cdf-45bb-a7fa-08b12663022c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.514817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 410/802] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:34 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/5ae58835c32e4a8f9e06f0a4c65ce26b/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:34.516139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.658643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a32a1-0cdf-45bb-a7fa-08b12663022c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.665513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f551fa-4104-4268-bc95-a40bdbd23bc7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.696256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.708697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.709580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.709986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.710324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.713207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.714557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.715022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.715465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.723132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.724832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.730144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.731964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d82bf469-8dab-4f77-b708-3815080c04d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:34.744493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:34.745605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:34.746494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:34.746494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:34.746494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:34.746494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 393/803] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:34.748179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.748179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.748351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:34.748925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:34.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:34.750734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:34.750734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:34.750734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:34.754341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 411/804] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:34.758165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.768597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:34.769712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:34.803972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.844672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 394/805] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:34.852912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.887535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6724109e-d1e1-4ca6-8605-bf5557f8c3e2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.951062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.953018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.953674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.954328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.956719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.958159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.962283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.991473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 412/806] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:34.994730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.061293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14910cc0-7959-4f7e-be76-919e2b689adb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.072814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.121158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.121781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.122055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.122325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.122984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:35.123287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.130565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.156361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 395/807] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.170788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.202872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.215015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600b884-7220-4ba4-86a9-7f71a8738813 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.257108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.258029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.258309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.258582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.266668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.268181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:35.282676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.283679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:35.284837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 413/808] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:35.296259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.303477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076ca80-e71c-477f-8922-c350aa8f8472 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.339882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.340415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.340699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.340992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.349330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.350792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:35.362692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.363861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 396/809] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:35.369633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.398151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.417258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5be76c-ad26-4bb2-a086-7132c6b513e2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.455142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d7546-60bb-4182-b009-646671a49995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.456135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7546-60bb-4182-b009-646671a49995 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.456583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7546-60bb-4182-b009-646671a49995 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.457124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d7546-60bb-4182-b009-646671a49995 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.457845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d7546-60bb-4182-b009-646671a49995 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.459374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d7546-60bb-4182-b009-646671a49995 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-826d7546-60bb-4182-b009-646671a49995 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.469409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:35.471602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:35.478496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c655d17-b2a8-49ab-9319-0e0218b9fd3e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.479436 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 414/810] 10.222.0.18 () {56 vars in 1129 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:35.480537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d7546-60bb-4182-b009-646671a49995 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.503520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.517886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.518512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.518800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.519076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.519949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:35.520205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.521845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.547990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 397/811] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.549254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.600665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d7546-60bb-4182-b009-646671a49995 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.636918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.637815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.638276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.638712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.639572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:35.640009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.641866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.645848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.662354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352d5b2-b824-40e6-a606-671d42ac6c0e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.669014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 415/812] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.670153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.699932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.700562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.701888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.701888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.701888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:35.702183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.703421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9434738-bc7c-40db-ae8a-0f53d072abd5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.716534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 398/813] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.716534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377bc8-850f-4186-a4bc-41b027715a9d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.716534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377bc8-850f-4186-a4bc-41b027715a9d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.716534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41377bc8-850f-4186-a4bc-41b027715a9d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.719353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 399/814] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:35.720628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377bc8-850f-4186-a4bc-41b027715a9d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.764927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.792318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5f0cb-4c8c-465c-9b14-e739b00a245a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.829700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.830265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.830539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.830806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.842908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.844387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.847653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41377bc8-850f-4186-a4bc-41b027715a9d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-25fb7993-2597-4190-a0e7-7d23c2e10603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:35.861183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.862243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:35.863141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:35.863141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:35.863141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:35.863141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 416/815] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:35.863847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.863847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.864092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.884404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:35.885651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:35.891295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.891943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.892238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.892504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.915903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.961596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 417/816] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:35.962569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.975281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.977371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.052162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.102808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 400/817] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:36.104113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.114315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588217e-1da3-42c2-8993-a212c12aafc6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.161364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.165583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.166197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.166467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.166730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.168982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.169363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.170812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.179639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 418/818] 10.222.0.18 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.181590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.223165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828e0eca-a3d9-4487-97bd-cc1b17ab301b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.258923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.259532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.259811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.260074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.260708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.261034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.262413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.276245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 401/819] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.278213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.302698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0523-583c-43a1-bcc8-327b2851062b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.325784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.341091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.341733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.342008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.342306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.353230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.354642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:36.369735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.370880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:36.371771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:36.371771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:36.371771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:36.375583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 419/820] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.376540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.386254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06816e4-bba5-46e4-a2a6-48e8f32a59f1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.425376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.426292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.426635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.426907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.427576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.428977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.460919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 402/821] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:36.460919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.492175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9846dd-0c55-4666-9eea-6e0132b96a79 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.542421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.543778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.544216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.544641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] PATH_INFO: `/v3/services/network` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.545431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.547011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.557651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.558748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:36.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:36.567645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 420/822] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.568961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.605657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.606619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c952981-5628-4ddd-b9a4-bf4555967cb3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.653432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.654187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.654526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.654859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.664398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.666262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:36.682895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.683890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:36.684778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 403/823] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.685864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.707493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fc4-4a94-45ae-a5b3-df2af954ee5f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.730928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.752792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.753446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.753720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.753989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.754630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.754940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.756294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.771191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 421/824] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.772226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.817904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13952ba7-962b-414b-b54d-7eb87cf64c89 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.854662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.855642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.856194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.856804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.857632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] RBAC: Adding query filter params (name=network) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.857935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.859489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.869654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 404/825] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.870659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.874701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.883512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2c0f7e-1422-43c3-bb10-262b64056b31 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.920150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.920767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.921083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.921344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.922032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.922331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.924365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe4a19-7906-4b89-890f-98f478c26d4f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.941071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 422/826] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.942538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a030-6005-4db9-a576-37a8401d296b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.942918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a030-6005-4db9-a576-37a8401d296b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.943285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96a030-6005-4db9-a576-37a8401d296b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.949173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 423/827] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:36.949173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96a030-6005-4db9-a576-37a8401d296b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.986312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.996986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4007c-0043-4c34-9aca-b3d9d42b2533 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.038273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.039059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.039338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.039604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.048169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.049806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:37.061824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.062955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:37.063860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 405/828] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.064383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.068672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96a030-6005-4db9-a576-37a8401d296b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.105312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.105925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.106252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.106459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.107143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] RBAC: Adding query filter params (type=network) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.107434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.108948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ee809-d7f7-4467-ad65-d56593c43b84 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.114358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 424/829] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.115567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.115844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.116106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.134785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:37.135961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:37.148398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.180631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7af98-32d7-4b07-9ed7-ac17654cdd51 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.207121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 425/830] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:37.207787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.222044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.223590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.224368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.224848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/be2777d5f726425e9d2c144b5d98f715/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.247834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=be2777d5f726425e9d2c144b5d98f715, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.249698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.264407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.277016 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 406/831] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:37 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/be2777d5f726425e9d2c144b5d98f715/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:37.278041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.328982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e18f4-8fb2-4447-8733-fabc2648d7dc None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.369821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.370903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.371398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.371861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.372865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.373415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.375254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.390688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 426/832] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.396342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.417291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.502661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dc2ba-8ca3-4b5c-8911-d39caaabf8ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.521810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d5c41-7782-400d-b9e7-8b7354602b87 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.545074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.546115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.546594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.547624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.548935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.551246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.564293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.564529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.564792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.564985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.565718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.566108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.568757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.576167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 427/833] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.577367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.585626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 407/834] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:37.713764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.749074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211da01d-7461-4bef-a9c5-603bb95464a0 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.809076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.809767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.810045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.810309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.825040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:37.830899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.831893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:37.832733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:37.834986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 428/835] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.841124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.901791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.957080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8d25e-0da8-4b43-ac32-c8e02ed2a3b8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.994833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.995521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.995914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.996275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.997209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.997601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.999139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.008746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 408/836] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:38.014559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.127729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96ac96-05cc-4966-8df9-8fa223878911 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.154347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.163760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.164424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.164694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.164990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.177053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.178514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:38.193620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.195853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:38.197382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:38.197382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.197382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:38.197382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 429/837] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.203544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.315835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1a872-ecf6-4d3d-94ca-9caae7c82977 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.334478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.352697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.353344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.353618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.353886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.354535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.354832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.356219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.369728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 409/838] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.378169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.489029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42379-8717-4e97-9f1c-291417408889 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.534586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.535236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.535515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.535770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.544646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.546125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:38.557861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.558922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:38.559953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 430/839] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.567350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.682158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fda70-b037-4375-bceb-e1bc3686fc94 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.718891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.719539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.719832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.720130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.720916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.721224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.722721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9a5e8-93b7-4282-8a13-e7fba39a64f7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.741833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 410/840] 10.222.0.18 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.750083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.759014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fe-f564-48ee-8989-4423da512d43 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.759345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fe-f564-48ee-8989-4423da512d43 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.759566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf178fe-f564-48ee-8989-4423da512d43 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.761091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 411/841] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:38.766441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.766720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.767059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.786952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:38.788185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:38.804648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.825247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.873126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 412/842] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:38.878698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11af93-20ce-43de-ad33-3dd3993322e8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.912952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.913571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.913847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.914112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.981722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.983270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.021277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.083030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 431/843] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:39.088756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.141616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.203002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032621d-a5d1-4ff5-9523-a20e4a95ab25 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.252027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.252677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.252982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.253408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.265464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c84d6-0024-4777-82f5-4113d6919da7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.267116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.269666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:39.288018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.289145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:39.290075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:39.290075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.290075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:39.290075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 413/844] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.297634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.304985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.305644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.305919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.306182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.306769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.308337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.317714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.319692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:39.320913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:39.321891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.321979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 432/845] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:39.329094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.332344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.437971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7b058-af25-4419-82a3-896d4b2dd90d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.448942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcd8ab3-4fc4-4e42-a981-562c67285142 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.481998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.482895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.483267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.483634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.484470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.484874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.486943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.487594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.488216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.488487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.489093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.489733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.490649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.492030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.499380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 433/846] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:39.503907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 414/847] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.504130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.512223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.580246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.618939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.619323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ddc8-006f-4aa5-a500-9c782a003fc8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.624919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d8bb8-52ea-4d95-a29e-558aaf346fdf None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.656070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.656700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.657013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.657283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.658087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.658292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.659679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.664311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.665062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 434/848] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:39.665453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.665744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.666057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.678543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.680399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4d6edb56-7009-4290-8d26-f627fb575b30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:39.694373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.695344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.696279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:39.699037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 415/849] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.705633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.831484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7cb63-bfb9-453b-a27a-19b30bb5694b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.865866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.873160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92882c31-380c-427b-b435-867b945d4774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.875051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92882c31-380c-427b-b435-867b945d4774 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.875588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92882c31-380c-427b-b435-867b945d4774 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.877253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92882c31-380c-427b-b435-867b945d4774 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.877857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5c7-4573-4e0d-b7c6-5a06d6d096dc None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.878301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92882c31-380c-427b-b435-867b945d4774 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.878301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92882c31-380c-427b-b435-867b945d4774 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.878850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5c7-4573-4e0d-b7c6-5a06d6d096dc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.879418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaae5c7-4573-4e0d-b7c6-5a06d6d096dc None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.879756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92882c31-380c-427b-b435-867b945d4774 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.881887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 416/850] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:39.887298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f3c0-ac00-4ff3-9994-34ca9af19ba1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.887596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f3c0-ac00-4ff3-9994-34ca9af19ba1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.887848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94f3c0-ac00-4ff3-9994-34ca9af19ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.895314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 435/851] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.901902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92882c31-380c-427b-b435-867b945d4774 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.917983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6a94f3c0-ac00-4ff3-9994-34ca9af19ba1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:39.919659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6a94f3c0-ac00-4ff3-9994-34ca9af19ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:39.995504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 417/852] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:40.024077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92882c31-380c-427b-b435-867b945d4774 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.061660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.062350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.062625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.062891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.063624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=5ae58835c32e4a8f9e06f0a4c65ce26b) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.064000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.065482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdeffe7-3926-4c43-b8c3-611e11f75617 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.072542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 436/853] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/role_assignments?user.id=5ae58835c32e4a8f9e06f0a4c65ce26b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:40.109339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1253b-022d-4955-b09d-0247d36fdc82 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.109713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1253b-022d-4955-b09d-0247d36fdc82 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.109885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1253b-022d-4955-b09d-0247d36fdc82 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.111671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 418/854] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:40.117608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.117942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.118141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.138259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:40.139478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:40.213292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.218369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 437/855] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:40.291880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1253b-022d-4955-b09d-0247d36fdc82 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.366536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.403821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1253b-022d-4955-b09d-0247d36fdc82 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.441007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.441652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.441924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.442283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.453495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:40.458435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.459567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:40.460610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:40.464637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 419/856] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:40.466112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.480739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.583630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba8735-bed8-4762-b19f-511dea178c1a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.598397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414019d9-7662-4b25-9457-4d9c3ca5835e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.624524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.625187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.625461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.625724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.626428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.626725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.628281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.631765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.643402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 438/857] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:40.645174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.647271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.648276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.648851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.649367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.650146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.651721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.661849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.662845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:40.663736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:40.671371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 420/858] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:40.673292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.698399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.758092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5e45a-32b5-42be-beb1-299ac9ed3e14 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.785387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38cb78e-fe8b-4b63-885e-de8496af431f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.794537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.795160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.795431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.795695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.807583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.809060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.825205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:40.825541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.826500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:40.827385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:40.827385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:40.827385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:40.828705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.829260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.829834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.830708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.831352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.831512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 439/859] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:40.832337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.833489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.844118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 421/860] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:40.845983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.913152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.952748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.959583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03287e5-0903-475d-b7ef-3b88252f3608 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.970583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15799391-0e55-42e8-9db2-67f52a4cd48e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.003258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.004554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.005579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.006068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.007155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:41.007776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.009693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.014263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.014941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.015219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.015489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.016175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:41.016478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.017968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.027233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 440/861] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:41.030492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 422/862] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:41.036929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.152136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63e61-a1f0-46c2-ab80-739c2c786dac None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.190944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.191609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.191879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.192150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.200603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.202175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:41.215689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:41.217483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:41.218544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 441/863] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:41.230076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.335702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec3635-3361-4809-b827-d3fab9610c27 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.358139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.373312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.373939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.374264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.374529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.375260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:41.375556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.377049 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.396221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 423/864] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:41.401931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.424283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.511491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b83e2c-9d33-411c-b741-2a4ebf49c202 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.547984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.548677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.549135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.549605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.636898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.638366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.657013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.749039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 442/865] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:41.754527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.865652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1452b10-d672-44bb-ab8f-a3ad6619e620 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.903308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.904340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.904831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.905365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.916629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.919274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b45db688-096d-486d-a4c3-9d6fa0ed1270 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:41.935176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:41.936206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:41.937151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:41.937151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:41.937151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:41.939459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 424/866] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:41.947145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.970648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d84ea-7912-4cfd-9cf3-8a1d18930bce None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.971299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d84ea-7912-4cfd-9cf3-8a1d18930bce None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.971711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d84ea-7912-4cfd-9cf3-8a1d18930bce None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.973823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 425/867] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:41.979683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.980170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.980676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.001204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:42.002865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:42.048417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.062398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75207585-2263-4f58-8cdc-96a84c746f59 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.087323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.112911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 426/868] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:42.118344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.118971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.119248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.119516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.120167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.120508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.122097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.135116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 443/869] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.140632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.245865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d34e31-19d2-45af-ba30-4dbfca4123c0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.282612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.283648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.284143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.284626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.292198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.294392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:42.308711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:42.309735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:42.310615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:42.313309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 427/870] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:42.319661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.394181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.419698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.433238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afca4c3-0677-4193-8c16-e659db0be054 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.468166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.468783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.469108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.469376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.470131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.470434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.471860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bf062-e1e8-43f1-9013-361e10693a55 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.490025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 444/871] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.490025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ac95-9600-4be4-88e4-1b293f50ece6 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.490025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ac95-9600-4be4-88e4-1b293f50ece6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.490025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39ac95-9600-4be4-88e4-1b293f50ece6 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.490025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 445/872] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:42.493334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39ac95-9600-4be4-88e4-1b293f50ece6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.510554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e47c-9d96-4d0d-8ca3-09ff98b12ee6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.543093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.555336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.556355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.557052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.557352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.558529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.561062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc782749-55ac-4f46-81ba-1b910c39693f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.581930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 428/873] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:42 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:42.583247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.583534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.583818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.604999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:42.606450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:42.630005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39ac95-9600-4be4-88e4-1b293f50ece6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.681938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.683604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.684204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.684660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.686039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=be2777d5f726425e9d2c144b5d98f715) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.687119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.689348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.694651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 429/874] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:42 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:42.698113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 446/875] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/role_assignments?user.id=be2777d5f726425e9d2c144b5d98f715&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.958643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.971748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.079734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfdce1-78f2-4636-a422-6c79abb65659 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.116323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.117069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.117348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.117617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.118205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.119720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:43.129340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:43.130402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:43.131235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 430/876] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:43.138066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.247356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.251396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4d319-2e05-4276-80a6-93d4a6772f00 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.287445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.288083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.288362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.288633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.289305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:43.289608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.290989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d727738-a53d-43d0-9d10-6f53bd8094f5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.296967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 447/877] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:43.302575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.316605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b21ad-85f6-48b7-aba7-42e5f5cc5290 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.316979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b21ad-85f6-48b7-aba7-42e5f5cc5290 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.317193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b21ad-85f6-48b7-aba7-42e5f5cc5290 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.318709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 448/878] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:43.324245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.324866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.324866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.346319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:43.347514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:43.402199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.424936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 449/879] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:43.445864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2f025-c282-48ce-8c71-157a8f57111a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.483652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.484282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.484560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.484826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.485545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:43.485848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.487232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9783cd-8458-4c8a-8db1-734fabd3d849 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.492680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 431/880] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:43.716210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.824497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.833005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab130f-3d49-4a8c-84fd-cffb9d77d239 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.868070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.868690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.868993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.869257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.869905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.871269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85461b75-33f5-4d2a-8ea4-b7a6f29bb9e3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.890305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 450/881] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:43 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:44.932982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbf5b-d302-413f-9e5e-05239cacc58e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.934016 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbf5b-d302-413f-9e5e-05239cacc58e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.934496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fbf5b-d302-413f-9e5e-05239cacc58e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.936548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 432/882] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:44.942567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c061dbb-6075-4917-a651-7e69903fbbd5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.943581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c061dbb-6075-4917-a651-7e69903fbbd5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.944134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c061dbb-6075-4917-a651-7e69903fbbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.964781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0c061dbb-6075-4917-a651-7e69903fbbd5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:44.966395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0c061dbb-6075-4917-a651-7e69903fbbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:45.042273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 451/883] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:45.070615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765089cd-0d7a-4b32-aba1-a41358674f67 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.071264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765089cd-0d7a-4b32-aba1-a41358674f67 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.071740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765089cd-0d7a-4b32-aba1-a41358674f67 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.073579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 433/884] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:45.079125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.079723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.080228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.099252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:45.100785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:45.176080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 452/885] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:45.328860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765089cd-0d7a-4b32-aba1-a41358674f67 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.433946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765089cd-0d7a-4b32-aba1-a41358674f67 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.442148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.455298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.470833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.471939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.472462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.472974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.473727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.475549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.485759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.486826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:45.487688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:45.490172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 434/886] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.495888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.498807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.567650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa881656-9027-4656-b9a0-4f079ac5a607 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.606094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198197-b078-4a3b-a067-edcffbb909ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.615761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.616790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.617701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.618141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.633780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:45.640435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.641968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:45.643041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:45.650612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 453/887] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.652311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.655618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.656265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.656532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.656798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.657686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.657881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.659852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.669798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 435/888] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:45.671286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.736159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.741978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.778307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a90565-60a1-4392-9fc5-efa47cea99a0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.780252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a92155-8fe5-4de0-8908-92ee73ebd782 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.814288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.815168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.815458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.815796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.816434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.816766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.818186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215ca64-dfb6-4b9e-874c-d2229f7a104b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.819448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.820235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.820486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.820746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.821427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.821721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.823056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a3f6e-c560-49f6-b438-da9fcdccc428 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.828421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 436/889] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.828421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e2cc-4e42-42aa-81a3-d5fbb0c2b9eb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.828421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e2cc-4e42-42aa-81a3-d5fbb0c2b9eb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.828421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2426e2cc-4e42-42aa-81a3-d5fbb0c2b9eb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.828421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 437/890] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:45.830250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e2cc-4e42-42aa-81a3-d5fbb0c2b9eb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.841240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 454/891] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:45.842669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.843309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.843601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.866295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:45.867401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:45.937993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 455/892] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:45.939182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.949582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2426e2cc-4e42-42aa-81a3-d5fbb0c2b9eb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.984140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.992645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.993397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.993765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.994095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.994880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] RBAC: Adding query filter params (interface=public, service_id=4d3b138492cb45dea35939d905c12c97, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.995261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.996747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.005719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 438/893] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/endpoints?service_id=4d3b138492cb45dea35939d905c12c97&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.011515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.041902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.097967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3a8dd-291d-471a-839c-bee995c1d868 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.127442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69273e1d-0947-42d0-a021-3052f6ef17ea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.140826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.141507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.141814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.142084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.152457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.153945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.160601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.161236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.161509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.161772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.162362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.163716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b56ee-4822-4ba1-9931-407d2f65f4a3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-f39e848e-f629-4e0e-b118-fd3b49ef73d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:46.169220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.170156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:46.171091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:46.171091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:46.171091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:46.171091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 456/894] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:46.171769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc285008-592a-4ea3-a508-c1776766b628 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.171769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc285008-592a-4ea3-a508-c1776766b628 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.172015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc285008-592a-4ea3-a508-c1776766b628 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.176936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 457/895] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:46.177432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 439/896] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:37:46.179438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc285008-592a-4ea3-a508-c1776766b628 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.183239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.183535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.183805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.205627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:46.206863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:46.266971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.283386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 440/897] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:46.284400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.309211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc285008-592a-4ea3-a508-c1776766b628 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.328474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.360993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.361610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.361880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.362143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.362787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.363085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.365775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.379302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 458/898] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:46.385058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.416305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72838f4-8032-4909-8eec-797b2828fa81 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.453852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.454532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.454806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.455071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.455712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.457133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.477148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 441/899] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:46 2022] POST /identity/v3/services => generated 241 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:46.514407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca0b44-4426-4859-93c3-9a4795d40deb None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.565586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.565952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.566634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.566913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.567178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.575835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.577434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:46.590821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.591865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:46.592814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 459/900] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:46.598101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.628675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.630081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.695483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1989143-6d3e-4593-b8fb-4bd09304196e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.708029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a7188-57d6-4761-a118-feae253bd331 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.742857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.743809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.744096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.744372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.745108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.746486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.749277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.750459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.750941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.751695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.752635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.753166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.754797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.757514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.758434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:46.759340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 442/901] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:46.765653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.781069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 460/902] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.788553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.881651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c69a6-37b6-4033-9e28-8881bbe89f78 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.886432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.896052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.911371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4773dd92-ee2c-4546-8989-f4bd284fe60d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.921385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.922329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.922329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.922642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.923287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.923583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.924988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.934856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 443/903] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.935954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.958620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.959240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.959509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.959771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.050097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.052326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.061325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87d8cd-1cc6-46d3-a314-a69dae1c8da0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.112973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.113609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.113882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.114149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.114803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.115087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.117267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.128919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 444/904] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:47.132158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.161112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.188608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 461/905] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:47.189359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.190430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.248196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26badce4-1cd3-4aa5-bbec-a348016413b6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.283107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.283498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.283768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.284031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.284725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] RBAC: Adding query filter params (interface=public, service_id=68073605ae8640a58a1f6aa1eb9b61b2, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.285054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.286430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.295461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 445/906] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/endpoints?service_id=68073605ae8640a58a1f6aa1eb9b61b2&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.300842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.321490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd26876-324a-43c4-9416-4735b3347e81 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.362350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.363164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.363371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.363643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.376681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.378192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:47.396918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:47.397924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:47.398794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:47.398794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:47.398794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:47.402942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 462/907] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:47.404242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.409837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ae39-6664-4722-a83c-470a43b6c9dc None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.413537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.453086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.454240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.454715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.455192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.456057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.456988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.457962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.469750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 446/908] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:37:47.529074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fd16c-cad9-404f-885e-13b79f5159fd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.569832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.570467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.570744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.571055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.571711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.572014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.573520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.587930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 463/909] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.593797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.705948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b33c7-17ca-4c03-826d-c05a3bf6503a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.741479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.742133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.742523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.742795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.750074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.751720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:47.763692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:47.764601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:47.765642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 447/910] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:47.771094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.875340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8c02d-b5a2-4dbd-b6f7-94240a3e2b79 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.877973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.911850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.912475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.912751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.913055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.913751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.914051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.915494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaca7c8-8929-48a9-87d6-9a98eee3be37 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.928544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 464/911] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.934557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.964733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.057284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b897d-98b0-462e-a79a-f255ecca40ef None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.094716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.095395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.095619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.095893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.096635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] RBAC: Adding query filter params (role.id=d0f50a7555e64269ac35fd2b72a6cbd1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=be2777d5f726425e9d2c144b5d98f715) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.097045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.098551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e575c3dd-0b93-4fc5-af73-54483ae18e3d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.105269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 448/912] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/role_assignments?user.id=be2777d5f726425e9d2c144b5d98f715&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=d0f50a7555e64269ac35fd2b72a6cbd1 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.392159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b91c-5f35-49e4-b2ce-23abed8d1987 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.392699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b91c-5f35-49e4-b2ce-23abed8d1987 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.392699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b91c-5f35-49e4-b2ce-23abed8d1987 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.394280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 465/913] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:48.400193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d2ab-86cd-4cdf-be7a-6639a382f661 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.400483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d2ab-86cd-4cdf-be7a-6639a382f661 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.400760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50d2ab-86cd-4cdf-be7a-6639a382f661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.420666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cf50d2ab-86cd-4cdf-be7a-6639a382f661 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:48.421916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cf50d2ab-86cd-4cdf-be7a-6639a382f661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:48.496854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 449/914] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:48.739434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dca0ef-ae53-4ec0-9ee7-e7ea07523688 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.739627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dca0ef-ae53-4ec0-9ee7-e7ea07523688 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.739907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dca0ef-ae53-4ec0-9ee7-e7ea07523688 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.741669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 466/915] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:48.747487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.747771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.748047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.766896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dca0ef-ae53-4ec0-9ee7-e7ea07523688 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.767827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:48.769086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:48.792750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.863785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 450/916] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:48.890569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dca0ef-ae53-4ec0-9ee7-e7ea07523688 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.928612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.929581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.929997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.930400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.931121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.932745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.943699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.945047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:37:48.946096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:48.948988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 467/917] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.957013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.080059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919daf1d-8ef3-43d8-a197-704df4f55882 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.118889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.119540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.119847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.120121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.120771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.121105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.122507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.128363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 451/918] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.134058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.148031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.168846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.171074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.254850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7919b799-3fd5-4dab-9232-d02203968694 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.267926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f468da-71fb-48a4-8e54-1e39eb7bd53d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.299664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.300822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.301309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.301960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.303161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.303896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.306079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.313663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.313889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 468/919] 10.222.0.18 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.318959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.319244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.319526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.319997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.320105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.321648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:49.331307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:49.332326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:49.333568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 452/920] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:49.342403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.414735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.432332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454989-8256-4fbb-acf4-21808e1dd7bd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.438029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.459346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed42f3-6756-4311-ac11-0572c6933058 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.469924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.470945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.471473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.471937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.472806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.474391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.497568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.498188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.498468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.498788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.499530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.499927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.501494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.505439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 469/921] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:49.511990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 453/922] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:49.513598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.635516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2706d2-7cbb-4d7f-bb60-35c78239072c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.679184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.679838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.680114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.680378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.681091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.681393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.682945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac44e56-4044-4304-af03-6a3d9b756e60 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.693041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 470/923] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:49.695824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.771668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f61b-5bb3-465e-8bde-82c26278d023 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.771978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f61b-5bb3-465e-8bde-82c26278d023 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.772209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266f61b-5bb3-465e-8bde-82c26278d023 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.773822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 471/924] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:49.779231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.779523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.779780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.786990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.806235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:49.807724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:49.816804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80066d85-3bdb-43f4-9c00-b38d3dfbfb8e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.853629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.854259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.854532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.854800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.855605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] RBAC: Adding query filter params (interface=public, service_id=62f6294adc844046a991424a56d4cc76, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.855913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.857323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.863576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 454/925] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/endpoints?service_id=62f6294adc844046a991424a56d4cc76&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.874101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.889792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.908256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 472/926] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:49.992624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efc57-14d9-48d8-aceb-7bf3db1ac0ff None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.028261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.028911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.029184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.029454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.030118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.031512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.045067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 455/927] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:37:50.200536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.316331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae363baa-0fb0-4a2a-b305-273ffd5b2ca7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.350939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.357730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.358653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.359150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.359610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.360494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.362266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-63ac8187-a4ec-41dd-9a51-17e2eeff855c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:50.371961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:50.373008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:37:50.373911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:50.376932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 473/928] 10.222.0.18 () {56 vars in 1132 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:50.383088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.430372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead5f9-169a-4389-9128-109554c62e94 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.430905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead5f9-169a-4389-9128-109554c62e94 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.430905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eead5f9-169a-4389-9128-109554c62e94 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.432935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 474/929] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:50.440059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.440612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.441093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.460990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:50.463326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:50.465767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.504245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e09002-14cb-414d-8ee3-50d54c767717 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.552876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 475/930] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:50.559211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.560083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.560557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.561038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.561841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.562385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.564017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.569810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 456/931] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:50.575701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.684675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6eaf4-a8c7-4c6d-98f0-500050a5cf40 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.711248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.720307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.721265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.721721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.722113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.722771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.723075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.724444 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.729973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 476/932] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:50.735812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.838106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d5b2c-9b5b-4ca9-af9d-fbd1f6a4b4f0 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.854054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.879109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.880190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.880674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.881200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.882076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.883733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.890317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.917881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 457/933] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:50.975353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ad80-bbce-422d-b89d-0391876249a5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.021740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.031496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.031794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.032075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.046018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:51.051114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.052099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:51.052971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 477/934] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:51.060451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.176383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2df3-f0e3-4274-8023-288867e83b6b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.212125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.212744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.213054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.213323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.213986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:51.214286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.215659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.224492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 458/935] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:51.230187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.281654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.340358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96deac0-efc0-4f0b-ae67-5ac566f25dfa None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.378432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.379092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.379372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.379643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.391430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.393067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:51.409097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.410092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:51.410969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:51.410969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:51.410969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:51.410969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 478/936] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:51.417999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.487644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.540644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b1670-16e0-4e6f-a70c-f0ac404efb92 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.576758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.577802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.578304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.578762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.579651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:51.580144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.581734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.597969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 459/937] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:51.604213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.725396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.729608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb01e1-a0d0-4ec7-b6d4-e995d509d894 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.764368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.764959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.765242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.765514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.773873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.775329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-2a382f33-70f0-40c3-a896-1761925f61f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:51.787112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.788095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:51.789048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 479/938] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:51.796567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.798568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a2669-aad8-4adc-b16a-ad761359ea7b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.798851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a2669-aad8-4adc-b16a-ad761359ea7b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.799124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14a2669-aad8-4adc-b16a-ad761359ea7b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.800929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 480/939] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:51.811969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.812265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.812522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.834695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:51.836214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:51.908348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c339ea2e-8d39-4b58-994c-0134c94ffad5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.924985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 481/940] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:51.930083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.955515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.956571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.957119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.957595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.958539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:51.959031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.960693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.981315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 460/941] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:51.992909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.094838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.113351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75203b6f-362b-4db3-88b4-d82379172097 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.154292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.154954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.155234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.155519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.168999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.170656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:52.187088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:52.188092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:52.189149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:52.189149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:52.189149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.189149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 482/942] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.194071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.244091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.312826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e6e72-6257-4a19-8207-fede0c4168e2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.351110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.352028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.352148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.352390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.353113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.353393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.354857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b166c4-a4ab-4560-b453-0b4c3d8c7dbd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.361626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0871d5-7248-4676-aba1-762117273e8d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.367547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 461/943] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.368861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033da7a2-fe98-4ca8-bf8d-69e7b02a7684 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.369020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033da7a2-fe98-4ca8-bf8d-69e7b02a7684 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.370845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033da7a2-fe98-4ca8-bf8d-69e7b02a7684 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.370845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 462/944] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:52.374880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033da7a2-fe98-4ca8-bf8d-69e7b02a7684 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.408671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.409311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.409586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.409867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.410502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.411982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.412375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.414742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING keystone.server.flask.common [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 19:37:52.426048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.433923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.sql.core [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:37:52.433923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:37:52.433923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [parameters: {'id': 'b3253ddf5348413ea9ac4c3bb428bede', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:37:52.433923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93090) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6bda9706-021d-4e51-855b-616d8867c072 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3253ddf5348413ea9ac4c3bb428bede.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3253ddf5348413ea9ac4c3bb428bede. Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:52.450051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 19:37:52.450941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application yield resource Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise self.value Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application session.commit() Sep 13 19:37:52.452008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:37:52.452984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:37:52.453895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:52.454746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [parameters: {'id': 'b3253ddf5348413ea9ac4c3bb428bede', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.455627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.456423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id b3253ddf5348413ea9ac4c3bb428bede. Sep 13 19:37:52.457666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.458179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 483/945] 10.222.0.18 () {60 vars in 1140 bytes} [Tue Sep 13 19:37:52 2022] POST /identity/v3/projects => generated 262 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.459350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.460031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.460031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.479137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:52.480630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:52.497084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033da7a2-fe98-4ca8-bf8d-69e7b02a7684 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.538683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.539319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.539589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.539850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.548248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.549883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.561134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 484/946] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:52.561134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:52.569209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:52.570143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.571047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:52.575521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 463/947] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.576804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.695617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56276fe4-7578-4eed-90f6-ed478bc2bd88 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.698386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.709023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d3368-7ead-4846-9fc0-267ef00d3204 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.727828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.739111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.739643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.739924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.740192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.748309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.748839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.749528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.749996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.750449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.750551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.751598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.752160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.753804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:52.763044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:52.763981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.764846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:52.769213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 485/948] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.770685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.771128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 464/949] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.778658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.884616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9964eb8-ab92-472b-833c-0eb5dec5696c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.900244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82557eb-8bf8-40b1-8b26-b6a10f6a5cf2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.926311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.926951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.927226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.927489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.928160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.928459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.929964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.938207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.939078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.939428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.939764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/be2777d5f726425e9d2c144b5d98f715/roles/d0f50a7555e64269ac35fd2b72a6cbd1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.942750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=be2777d5f726425e9d2c144b5d98f715, role_id=d0f50a7555e64269ac35fd2b72a6cbd1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.944289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.957004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 486/950] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.957004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.977022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 465/951] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:52 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/be2777d5f726425e9d2c144b5d98f715/roles/d0f50a7555e64269ac35fd2b72a6cbd1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:53.033939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.087568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad53bc-327c-456e-9988-c4acdb6e61fe None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.129026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.129829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.130110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.130379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.130996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.132665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-01c3cf8b-cc6b-4c42-840a-dab754e728e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:53.143199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:53.144276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:53.145399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 487/952] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:53.153620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.174875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.277630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3387a3-97fc-44ee-8431-ec8527fb3f75 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.293125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db563-92d3-45a7-a511-58c101da3803 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.293489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db563-92d3-45a7-a511-58c101da3803 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.293781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8db563-92d3-45a7-a511-58c101da3803 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.295147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 488/953] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:53.300586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.300999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.301338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.322011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.322840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.323116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.323377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.324010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:53.324302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.325570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:53.325774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.326781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:53.332552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 466/954] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:53.338339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.416682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 489/955] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:53.441278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.466644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7139482-759c-4450-826d-e6c0e00b0ec1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.501471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.502108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.502351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.502616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.503243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:53.503540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.504922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.510763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 467/956] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:53.519704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.564803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.630145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf0436-ff9a-4d9d-916d-7b0524267c59 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.672434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.673068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.673341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.673602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.674272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.675649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.698103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.699588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 490/957] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:37:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:53.823701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d720d6-436d-4e4e-ab2f-e14f74222bda None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.876407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.877094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.877375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.877643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.878232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.879640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.882859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:53.894172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:53.895302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:53.896227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:53.898441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 468/958] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:53.904681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.009838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:54.019662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6336d13e-f404-4aee-a90b-42bbe528b016 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.056251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.056880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.057191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.057463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.058103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] RBAC: Adding query filter params (name=compute) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:54.058406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:54.059811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a935a96-3391-42f3-be3b-cad8519e770b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:54.066167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 491/959] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:54.072249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.176689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb6c82-0762-40be-b4c4-671d8963b153 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.212010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.213102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.213577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.214038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.214888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:54.215377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:54.216955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7100480-bcc4-4567-9602-1370e0fdf239 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:54.222048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 469/960] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:55.326348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8298f-c7c9-4328-8c0f-c71ec1a73485 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.326893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8298f-c7c9-4328-8c0f-c71ec1a73485 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.326893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8298f-c7c9-4328-8c0f-c71ec1a73485 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.328486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 492/961] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:55.333905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.334332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.334432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.348365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35807e1-f6db-4f7f-841c-d763f67fa054 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.348685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35807e1-f6db-4f7f-841c-d763f67fa054 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.348936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35807e1-f6db-4f7f-841c-d763f67fa054 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.350611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 493/962] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:55.353403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:55.354760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:55.356041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.356333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.356607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.375375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:55.376631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:55.426677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.438828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 470/963] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:55.462338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 494/964] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:55.704266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.729033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.819609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2c592-3ae9-479e-8462-b828de4d473e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.850988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.858790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.860006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.860330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.860558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.867866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97201c03-1ee0-44b3-93ac-d740a73e03a7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.874062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:55.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:55.880767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:37:55.881858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:55.888783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 471/965] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:55.890941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.917648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.918278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.918555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.918870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.919534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.920993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.937143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.sql.core [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:37:55.937143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:37:55.937143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [parameters: {'user_id': 'a6896f27c803440ba8c526c859a32909', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:37:55.937143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93090) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:55.944293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:55.945247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application yield resource Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise self.value Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application session.commit() Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:37:55.946217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:37:55.947649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:55.948556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:55.949488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a6896f27c803440ba8c526c859a32909', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:55.950333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 19:37:55.951142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:55.952130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 495/966] 10.222.0.18 () {60 vars in 1131 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/users => generated 160 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:55.953513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.968763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.019393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52c1b8-96ed-40d3-8e0a-a996f49191af None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.060769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.061530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.061865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.062136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.062850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.063153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.064638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f5584a-8caf-422f-8669-37739f6e4346 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.072680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3c10d-ccfd-4474-8f51-f8e75b7aed81 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.073645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 472/967] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.078411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ed129-b380-4f13-98e7-88ba2109a0db None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.079014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ed129-b380-4f13-98e7-88ba2109a0db None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.079537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ed129-b380-4f13-98e7-88ba2109a0db None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.081449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 473/968] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:56.083183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ed129-b380-4f13-98e7-88ba2109a0db None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.112078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.112659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.112973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.113252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.121293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.123306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-618f858b-7888-4381-af4e-fa3739f2c61e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:56.135747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.136770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:56.137792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 496/969] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:56.139039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.139039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.139039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.157703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:56.159333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:56.169233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.200252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ed129-b380-4f13-98e7-88ba2109a0db None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.217491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.237083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 497/970] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:56.237773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.245027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.246006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.246935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.247409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.262755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.265418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:56.286367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.287607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:56.288728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:56.288728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:56.288728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:56.291362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 474/971] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:56.298544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.396089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6790140-f9ba-4d9b-9da4-86fd8327e3db None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.433510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a655e07-a4a9-4bd7-a061-7ee6017182b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.448419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.449771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.450234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.450992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.452270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.452791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.455215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0632d-572c-4932-9771-383f0a4f9797 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.481566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.482181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.482457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.482722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.483369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.483661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.485064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd019e4-a2c4-40f2-b925-2bb2256a4a5f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.489740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.497195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 498/972] 10.222.0.18 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.504978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 475/973] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.505247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f6c-93b9-45ec-86ad-e550a2180e96 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.505247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f6c-93b9-45ec-86ad-e550a2180e96 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.505247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b65f6c-93b9-45ec-86ad-e550a2180e96 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.505247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 499/974] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:56.510656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.510916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.511252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b65f6c-93b9-45ec-86ad-e550a2180e96 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.511407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.532413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:56.534068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:56.563226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.615478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 476/975] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:56.616459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.655020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b65f6c-93b9-45ec-86ad-e550a2180e96 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.699533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.700452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.700824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.701238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.710725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.712388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:56.726278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.727285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:56.728215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 500/976] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:56.733133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.762114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e24727f-20f9-4c1d-9030-3cd3ba67bb7e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.812223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.814216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.815070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.816948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.816948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.816948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.818137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.832276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.833443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:56.834281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 477/977] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:56.839768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.862777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece65c3-35b7-437b-bf39-dd64a738f2e3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.870092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.899482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.900094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.900368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.900631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.901386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.901686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.903105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.921701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 501/978] 10.222.0.18 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.922556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.948717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2096fe2e-f231-4d8d-a7b0-e0c9d6eb75b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.982849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.983466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.983749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.984022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.984656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.984987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.986376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.991425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 478/979] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:56.992224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.030472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30072fc1-6a37-4fca-b9ee-d67253ac5ef2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.049158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.066267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.066883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.067156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.067418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.068063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.069472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.080195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.085465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 502/980] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:57.086454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.096645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b10ecc0-5548-42ba-9fa9-d2d86ad21dbd None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.142722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.143356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.143636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.143908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.207531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79239c0-3d49-43da-9dd2-98956bc5c9c6 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.221157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.222660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.248327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.249137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.249418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.249684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.250456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.250793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.252225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.260591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 503/981] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:57.266079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.315534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.323205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.348988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 479/982] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:57.352454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.378277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b37faa-600e-47f2-b6b4-64747041fff1 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.421883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.422770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.423068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.424284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.425063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] RBAC: Adding query filter params (interface=admin, service_id=62f6294adc844046a991424a56d4cc76, region_id=RegionOne) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.425351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.426740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.432307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 504/983] 10.222.0.18 () {56 vars in 1249 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/endpoints?service_id=62f6294adc844046a991424a56d4cc76&interface=admin®ion_id=RegionOne => generated 193 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:57.438120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.456328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44dd0b-e87b-4554-ace1-3bbd3e184b91 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.505619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.506546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.506969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.507396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.518288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.520040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:57.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:57.538347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:57.539491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:57.539491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:57.539491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:57.545906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 480/984] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:57.547354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.559193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fdf59-bb0c-4231-8e79-6cedfbde8772 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.568355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.587361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.606459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.607095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.607373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.607644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.608295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.609713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.624385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 505/985] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:37:57.660906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4096e2-c688-45ef-a804-fdd2d333e84f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.702262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.702901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.703176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.703438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.704125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.704426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.705869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.719615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 481/986] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:57.725384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.841749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faee17d-bf01-4ab9-9487-8202ecc5ea3a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.878161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.878810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.879049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.879342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.886863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.888323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:57.901036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:57.902006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:37:57.902917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 506/987] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:57.908035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.009558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.018820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda7f78-eb3c-414b-a594-1310c808fe04 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.057865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.058478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.058754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.059020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.059709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.060005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.061461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33191c8-02ac-4ce5-b8d2-5d7bdb33b89e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.074784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 482/988] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:58.080748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.122556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.212608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b8434-1ad6-4ef2-ab29-616480e4643e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.251941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.252553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.252829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.253124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.253840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] RBAC: Adding query filter params (role.id=d0f50a7555e64269ac35fd2b72a6cbd1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=be2777d5f726425e9d2c144b5d98f715) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.254144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.255574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3d74b1-883f-4ca0-aa39-3fd70701a85f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.264932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 507/989] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/role_assignments?user.id=be2777d5f726425e9d2c144b5d98f715&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=d0f50a7555e64269ac35fd2b72a6cbd1 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:59.287444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd75c-7c0b-484a-b91f-8e6c159ddb10 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.287988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd75c-7c0b-484a-b91f-8e6c159ddb10 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.287988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcd75c-7c0b-484a-b91f-8e6c159ddb10 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.289242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 483/990] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:59.295338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8bc21-ab92-451c-8d06-91691e2efb92 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.295828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8bc21-ab92-451c-8d06-91691e2efb92 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.295828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d8bc21-ab92-451c-8d06-91691e2efb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.315127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-38d8bc21-ab92-451c-8d06-91691e2efb92 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:59.316324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-38d8bc21-ab92-451c-8d06-91691e2efb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:59.387144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 508/991] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:59.652330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcd75c-7c0b-484a-b91f-8e6c159ddb10 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.765693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcd75c-7c0b-484a-b91f-8e6c159ddb10 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.798476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d0df-13ce-401e-82b9-7d5f32c6ec0b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.798759 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d0df-13ce-401e-82b9-7d5f32c6ec0b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.799020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2d0df-13ce-401e-82b9-7d5f32c6ec0b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.800376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 509/992] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:59.805398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.805792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.806041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.806140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.806317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.806425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.806606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.807186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.808658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:59.818173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:59.819206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:37:59.820055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 484/993] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:59.825363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.825644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:59.826944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:59.846673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.849979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.910096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 510/994] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:37:59.940914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f984d9-1e2f-49fd-a4cb-cc5702d3bf54 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.978184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.978812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.979075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.979339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.979978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] RBAC: Adding query filter params (name=compute) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:59.980277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.981671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.987023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 485/995] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:59.992662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.110598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dfbb1-5a13-4440-a74f-1664cd30642c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.146764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.147398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.147676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.147940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.148571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] RBAC: Adding query filter params (type=compute) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.148875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.150282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.159895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 511/996] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.172879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.175010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.194279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.272741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.298777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b397de-4e02-4b03-9ccd-82486554324a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.306246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd5dd4-58f1-47e4-8c3c-c72d2c630b9e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.336019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.336717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.337032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.337434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.338257 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] RBAC: Adding query filter params (interface=public, service_id=7a77c37176a2435b9e4b3f153e642136, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.338566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.339985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.343647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.344301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.344616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.344908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.345520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.350786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 486/997] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/endpoints?service_id=7a77c37176a2435b9e4b3f153e642136&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.351020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.352027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:00.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:00.360701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:00.366767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 512/998] 10.222.0.18 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:00.367617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.454855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.467235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42694c60-e21c-44aa-8f76-ab1954a33581 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.475070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba2e06-b175-4d37-87e9-023bf2772e5f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.502844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.503463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.503742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.504009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.504618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.506029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a9a7b-2eb9-4588-ba50-86ca85de1996 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.509100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.509714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.509990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.510260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.510885 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.511185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.512424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 487/999] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:38:00.512513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.513673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f4156-acd0-431c-acd6-3481f834688f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.513956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f4156-acd0-431c-acd6-3481f834688f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.514231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f4156-acd0-431c-acd6-3481f834688f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.516938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 488/1000] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:00.518108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 513/1001] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.522382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.522929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.523464 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.530903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.543793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:00.545180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:00.549564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.639449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 489/1002] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:00.677513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e52eab-2f05-4041-93b3-c26898344408 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.717388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.717934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.718206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.718473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.719106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.719423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.721827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.727421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 514/1003] 10.222.0.18 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:00.733384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.845185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24998fa8-344f-4394-a7bf-7325de4b8bdc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.881687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.882718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.883203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.883666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.884402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.885828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.899872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.907197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 490/1004] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:00.911259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.932738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.061425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b77124d-ff66-42f2-81a5-4d93775e5fe4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.112217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.112947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.113248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.113517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] PATH_INFO: `/v3/services/image` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.114125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.115539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:01.125615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:01.126750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:01.127764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 515/1005] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services/image => generated 86 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:01.133173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.244355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce186ab-034f-49aa-b7ac-f5c446064d3c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.289209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.289751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.290028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.290300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.290960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] RBAC: Adding query filter params (name=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.291290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.293043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5fe5c-b7ed-4e25-a611-812406a0f000 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.298804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 491/1006] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.304801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.388853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.423389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a55da-8606-41f6-bd2f-a9b098711e54 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.463030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.464095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.464672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.465167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.466009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] RBAC: Adding query filter params (type=image) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.466499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.468095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8f48fb-75d9-43fd-bd8c-8c0ab879a992 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.474331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 516/1007] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:02.743364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a73c-eb6e-43e8-9157-0cf329a6d11a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.743908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a73c-eb6e-43e8-9157-0cf329a6d11a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.743908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb8a73c-eb6e-43e8-9157-0cf329a6d11a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.745512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 492/1008] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:02.751012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4f213-7f06-4ee5-96b6-c381a050c698 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.751585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4f213-7f06-4ee5-96b6-c381a050c698 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.751585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4f213-7f06-4ee5-96b6-c381a050c698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.770121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-11e4f213-7f06-4ee5-96b6-c381a050c698 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:02.771314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-11e4f213-7f06-4ee5-96b6-c381a050c698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:02.843355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 517/1009] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:02 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:03.104732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8a73c-eb6e-43e8-9157-0cf329a6d11a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.108876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff09f7-070f-44cb-8630-436acd8c1755 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.109184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff09f7-070f-44cb-8630-436acd8c1755 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.109453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ff09f7-070f-44cb-8630-436acd8c1755 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.110928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 518/1010] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:03.116445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.116445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.116791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.135211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:03.136437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:03.188847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.208335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.220955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 519/1011] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:03.244629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb8a73c-eb6e-43e8-9157-0cf329a6d11a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.281687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.282414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.282636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.282901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] PATH_INFO: `/v3/services/compute` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.283491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.284899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:03.294934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:03.295997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:03.297989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 493/1012] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:03.302868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.431851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5735bc-3443-4b91-a55a-5630ce4d7b37 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.470064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.470686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.470964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.471239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.471873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] RBAC: Adding query filter params (name=compute) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:03.472175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.473593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.478872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 520/1013] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:03.484473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.488054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.561558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.611458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5b875-2828-4662-8415-92d96acdad7b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.619321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9e717-e5d9-4dc0-b575-8dd59b61b69c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.654969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.664668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.664874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.665314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.665413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.665587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.665672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.665847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.665930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.666478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:03.666772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.667367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.668493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644c088-eb17-4a57-baaa-3d6c43e859e4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.669360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0f545b-4419-49e3-ba33-761d2912d3b3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.670381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 521/1014] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:03.671594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce61b-52a8-47d8-9771-47f8eaba6c5b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.671872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce61b-52a8-47d8-9771-47f8eaba6c5b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.672135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ce61b-52a8-47d8-9771-47f8eaba6c5b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.673547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 522/1015] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:03.676137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ce61b-52a8-47d8-9771-47f8eaba6c5b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.676782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 494/1016] 10.222.0.18 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:03.679161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.679446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.679722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.697675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:03.698867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:03.775287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 495/1017] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:03 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:03.776116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.787077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ce61b-52a8-47d8-9771-47f8eaba6c5b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.815137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.826605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.827225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.827502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.827767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.828841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.830677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.872930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 523/1018] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:03 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:03.903554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2f42-6148-4df8-9d8e-735f23fe87fc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.934306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.949807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.950437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.950710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.950976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.951651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.953102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb64ef-c877-472f-9530-dee8afe912af None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.979670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 496/1019] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:04.045082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.162750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f437599-cfea-4c5c-83aa-be8cddda7366 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.205395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.206016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.206288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.206551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.207196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.208558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c73aad6-ee26-493d-aed7-c77104842fbe None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.226141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 524/1020] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:04 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:06.110516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d680ee-1a7d-410d-a829-d3a6825a9540 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.111067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d680ee-1a7d-410d-a829-d3a6825a9540 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.111067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d680ee-1a7d-410d-a829-d3a6825a9540 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.112429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 497/1021] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:06.118365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c6dba-9de6-443b-81a2-4a0a16c666e2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.118868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c6dba-9de6-443b-81a2-4a0a16c666e2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.118868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c6dba-9de6-443b-81a2-4a0a16c666e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.139487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-885c6dba-9de6-443b-81a2-4a0a16c666e2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:06.140735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-885c6dba-9de6-443b-81a2-4a0a16c666e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:06.174232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.228927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 525/1022] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:06.251386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e8ca-e011-470a-bd7d-c6593cfe6511 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.251715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e8ca-e011-470a-bd7d-c6593cfe6511 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.251932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4494e8ca-e011-470a-bd7d-c6593cfe6511 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.253325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 498/1023] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:06.258845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0204b4-1cf9-4f13-a6a8-b359d89b31ce None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.259169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0204b4-1cf9-4f13-a6a8-b359d89b31ce None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.259390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0204b4-1cf9-4f13-a6a8-b359d89b31ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.279698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bd0204b4-1cf9-4f13-a6a8-b359d89b31ce None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:06.280930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bd0204b4-1cf9-4f13-a6a8-b359d89b31ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:06.373132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 526/1024] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:06.388921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1573d-f64d-4ec8-83d9-3976a9817e8a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.389169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1573d-f64d-4ec8-83d9-3976a9817e8a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.389467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1573d-f64d-4ec8-83d9-3976a9817e8a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.390796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 499/1025] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:06.396312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.396598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.396867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.416413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:06.417982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:06.483600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 527/1026] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:06.524467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1573d-f64d-4ec8-83d9-3976a9817e8a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.631763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1573d-f64d-4ec8-83d9-3976a9817e8a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.637232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.659833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.670105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.670761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.671234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.671499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.672707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.674206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.676213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 500/1027] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:06.684192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.733153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.762446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b31682-56dd-4872-a1a9-a50413233da2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.789120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfed90-abd2-46fd-adca-2068af6ca516 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.796733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.797407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.797733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.798007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.808032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:06.812840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:06.814053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 528/1028] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:06.814953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.825526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.826133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.826409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.826675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.827328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.828758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.851043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 501/1029] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:06 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:06.852261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.913446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.938849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f68adb-9f46-4cd0-a767-ee4bb770f9bf None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.978374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.979191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.979472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.979739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] PATH_INFO: `/v3/services/image` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.980338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.980997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fa8e69-cdfd-482e-a21c-2b32dd5ca84f None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.981814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.982922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:06.992826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:06.993961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:06.994825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 529/1030] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:07.000650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.030037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.030985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.030985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.031272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.031991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.032293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.033752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.044152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 502/1031] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:07.050006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.129382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8709107-495f-48a7-8c2e-2216d573a1e0 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.165062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374d0fb-4ba5-4777-9ed5-939ff49e0fd2 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.165594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.166095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.166528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.166809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.168638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] RBAC: Adding query filter params (name=image) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.168968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.170361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.177132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 530/1032] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:07.182535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.195500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.206029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.206669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.206948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.207218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.218369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.219891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:07.234422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:07.235482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:07.237219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:07.237219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:07.237219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:07.237219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 503/1033] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:07.241452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.261279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.301138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26612ff2-9aa3-4210-ac1f-c88a3276ccba None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.337466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.338088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.338367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.338641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.339258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] RBAC: Adding query filter params (type=image) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.339815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.340972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.346360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 531/1034] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:07.352176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.353985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc72990-b117-4517-9af5-9c81dfe9456d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.391229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.391835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.392117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.392389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.393072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.393375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.394877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.407116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 504/1035] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:07.412735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.430565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.462921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f973b0-8231-45ba-b3ad-d0615d3cd4df None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.495216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.499511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.500204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.500878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.501191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.502437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] RBAC: Adding query filter params (interface=public, service_id=45593c474a974d99a41919244ce6cf1e, region_id=RegionOne) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.502745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.504273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.510437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 532/1036] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/endpoints?service_id=45593c474a974d99a41919244ce6cf1e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:07.516163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.528131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc22628-bc0e-4381-835f-719bb0c7afc8 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.567758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.568547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.568826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.569130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.577235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.578722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:07.591045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:07.592089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:07.593786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 505/1037] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:07.598090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.630477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108db5d5-6193-455c-8bcd-b429a09497fd None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.667521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.668140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.668414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.668708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.669318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.670791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.672935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.680069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 533/1038] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:38:07.707705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54151a46-cf62-4618-9242-5de91c9719b7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.749534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.750180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.750412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.750805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.751569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.751867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.753369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.756794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.777121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 506/1039] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:07.783115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.900055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eea87b-5007-4709-b256-81a46cb6dd4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.937569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.938351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.938601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.938864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.005520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.008817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.010730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.117588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 534/1040] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:08.123276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.227386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031e8e3-9dd7-4084-af2f-59f4aef8fa6d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.262321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.262869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.263137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.263402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.273755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.275184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:08.289373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:08.290322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:08.291258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:08.291258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:08.291258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:08.292341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 507/1041] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:08.296602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.393403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.396775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930c235-50c2-470d-8939-002b32ec4d77 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.431598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.432206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.432475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.432738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.433399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.433693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.435084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.447199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 535/1042] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.452728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.499734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.554860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62532-66e5-4fd0-81f4-fa8b35642752 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.595258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.595914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.596189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.596454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.604295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.605786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:08.617526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:08.618438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:08.619450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:08.620512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 508/1043] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:08.624529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.729140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad74235c-8960-467a-bcd7-794d86b51f6d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.762458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.763080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.763359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.763627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.764315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.764616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.766164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7044a35a-86cf-43a6-bfe3-f0aab029986b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.779505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 536/1044] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.785594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.887189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8c16b-3be8-4874-986d-da208d78883b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.894762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.925513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.926150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.926427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.926695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.927444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] RBAC: Adding query filter params (role.id=d0f50a7555e64269ac35fd2b72a6cbd1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.927749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.929276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489a62c-05f5-410e-9f88-9121637754cf None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.935996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 509/1045] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=d0f50a7555e64269ac35fd2b72a6cbd1 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.051189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331d1e-3b10-41ff-ac90-e9f1e0c9dea7 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.051448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331d1e-3b10-41ff-ac90-e9f1e0c9dea7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.051685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04331d1e-3b10-41ff-ac90-e9f1e0c9dea7 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.053301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 537/1046] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:09.058963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.059292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.059491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.079735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:09.080947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:09.161063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 510/1047] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:09.417746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04331d1e-3b10-41ff-ac90-e9f1e0c9dea7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.440825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.523656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04331d1e-3b10-41ff-ac90-e9f1e0c9dea7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.557904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.558534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.558807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.559070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.569544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:09.574315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:09.575301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:09.581581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 538/1048] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:09.582348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.678082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.681978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27840d73-513c-40d1-a8f2-09be6a611968 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.716436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.717081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.717354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.717616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.718282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:09.718574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.720142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf18cd-b847-4b92-a456-181088e1a2e3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.730079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 511/1049] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:09.736188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.824865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.845865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e1991-9fff-42c2-9350-2f58538cdc07 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.849532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98312-e80a-4311-a808-b723b26562bd None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.849821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98312-e80a-4311-a808-b723b26562bd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.850094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d98312-e80a-4311-a808-b723b26562bd None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.851564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 512/1050] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:09.856425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.856710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.857222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.878950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:09.880479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:09.881348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.881986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.882262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.882531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] PATH_INFO: `/v3/users/9e280adb2a2e491e92e5e59f648e799d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.897963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] RBAC: Authorizing `identity:get_user(user_id=9e280adb2a2e491e92e5e59f648e799d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.899407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.905637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 539/1051] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/users/9e280adb2a2e491e92e5e59f648e799d => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.906768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.957690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 513/1052] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:10.010627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cfb98-644d-4165-b231-0845ffdb02fc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.054221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.054836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.055106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.055372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.064685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.124972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.126576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.222899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.234031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 540/1053] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:10.235153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.275724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.321954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.331798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41e70-1171-4a66-87d5-51f544d2e6b8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.342961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e6d23-2362-424c-b09a-ad7858c1ad96 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.368465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.369128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.369512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.369784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] PATH_INFO: `/v3/services/image` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.370479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.371844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:10.381819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:10.382859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:10.384380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 514/1054] 10.222.0.18 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:10.385472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.385472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.385472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.385472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.386625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.388118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.388768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.389962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 541/1055] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:10.395730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.489630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32ff34-9ccc-4431-939b-c5bc7702a888 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.503086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51030f-cc7b-433d-889e-9decccdd385f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.510671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.524456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.525123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.525409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.525675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.526301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] RBAC: Adding query filter params (name=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:10.526602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.528065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.539530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 515/1056] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:10.540921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.541379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.542335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.542633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.542867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.543610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=31b6ce5375d84c4682351a53d1d5a340, user.id=9e280adb2a2e491e92e5e59f648e799d) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:10.543895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.545331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.551199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 542/1057] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/role_assignments?user.id=9e280adb2a2e491e92e5e59f648e799d&scope.project.id=31b6ce5375d84c4682351a53d1d5a340&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:10.567263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.656785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700d2d4-82fb-4c3b-9fdb-47359b6e7441 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.695223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.696171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.696647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.697141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.698093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] RBAC: Adding query filter params (type=image) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:10.698586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.700239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434930c-ba27-4de1-8a3c-09944c30f87a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.706595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 516/1058] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:10.712644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.819340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c933e-7226-47e2-9a85-e609234bcc11 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.854128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.855064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.855527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.855975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.856830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.858423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d5f9b-e688-41ea-b544-4aed8e4559f6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.881234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 543/1059] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:11.057479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae42e-e0c9-4733-ac11-6015d6c5845b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.057479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae42e-e0c9-4733-ac11-6015d6c5845b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.057874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eae42e-e0c9-4733-ac11-6015d6c5845b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.059798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 517/1060] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:11.069953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152658a2-3096-4473-b685-a643d8102400 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.070127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152658a2-3096-4473-b685-a643d8102400 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.070390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152658a2-3096-4473-b685-a643d8102400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.095122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-152658a2-3096-4473-b685-a643d8102400 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:11.099986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-152658a2-3096-4473-b685-a643d8102400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:11.122183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.182382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 544/1061] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:11.445529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eae42e-e0c9-4733-ac11-6015d6c5845b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.532370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.551623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eae42e-e0c9-4733-ac11-6015d6c5845b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.586584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.587210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.587483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.587746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.598072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:11.602770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:11.604453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:11.606595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:11.609067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 518/1062] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:11.610434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152658a2-3096-4473-b685-a643d8102400 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.712482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152658a2-3096-4473-b685-a643d8102400 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.748029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.748660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.749033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.749305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.749972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:11.750278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.751742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.760569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 545/1063] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:11.766221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.875916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43f544-c72d-4b1e-8b1b-853974bcb914 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.911347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.912002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.912282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.912548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.923287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.924747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.927323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:11.942087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:11.943131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:11.944036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:11.944036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:11.944036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:11.945243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 519/1064] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:11.949271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.966769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.064643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc25b93-0076-474f-b134-562bd51216cc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.101372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.102009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.102290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.102564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.103223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:12.103521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.104966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.120055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 546/1065] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:12.125901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.238623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595a20a3-3fe3-4e35-87b1-3077a732fab9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.273389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.274069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.274346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.274614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.282505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.283956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:12.295692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:12.296662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:12.297763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:12.301247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 520/1066] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:12.302819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.381754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.407636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277e341-1dd0-42a6-aa04-2700f443736e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.443173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.443810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.444088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.444356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.445111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:12.445413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.446837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.464450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 547/1067] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:12.470232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.528393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.579378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c6ad-20d0-4696-90cc-524d0ab50bac None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.616407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e042520-ac03-496e-9379-850373528e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.617079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e042520-ac03-496e-9379-850373528e0e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.617359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e042520-ac03-496e-9379-850373528e0e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.617628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e042520-ac03-496e-9379-850373528e0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.628727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e042520-ac03-496e-9379-850373528e0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.630208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e042520-ac03-496e-9379-850373528e0e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0e042520-ac03-496e-9379-850373528e0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:12.645099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:12.646161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:12.647071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:12.647071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:12.647071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:12.649265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 521/1068] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:12.652032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.685925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09a2f-cf97-4e28-bdc1-f9c555934a42 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.686240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09a2f-cf97-4e28-bdc1-f9c555934a42 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.686491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a09a2f-cf97-4e28-bdc1-f9c555934a42 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.687992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 522/1069] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:12.694272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.694569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.694823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.714075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:12.715271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:12.759090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8f01-b36a-4d80-b5f5-fda169389525 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.788986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 523/1070] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:12.790697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.797684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.798317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.798598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.798892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.799512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:12.799812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.801219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.813251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 548/1071] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:12.818979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.859439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.920470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d849474-b897-4c67-9142-ffcf7434de4e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.957468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.965151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.965151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.965151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.971357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.972846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:12.985366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:12.986429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:12.987442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:12.989341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 524/1072] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:12.992583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.051958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.108350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756adb-2ef5-403e-a86f-dd8ad52c2b4d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.148932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.149915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.150199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.150739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.151452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:13.151744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.153247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149071b4-404b-4867-9f00-d99f8955faab None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.155366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.160133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922bbe26-83d4-43ea-af0f-75bd5b9828aa None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.176215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 549/1073] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:13.177651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c458-f7bb-4a60-8f4e-0c6c71199e81 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.177937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c458-f7bb-4a60-8f4e-0c6c71199e81 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.178213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291c458-f7bb-4a60-8f4e-0c6c71199e81 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.179834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 550/1074] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:13.180769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291c458-f7bb-4a60-8f4e-0c6c71199e81 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.205343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.206093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.206342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.206622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.218723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-8cd22beb-d7e7-41a0-b9ba-2c6f0e70f7bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:13.223967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:13.224980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:13.225944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 525/1075] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:13.226980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.227058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.227307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.245561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.250295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:13.251678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:13.296252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291c458-f7bb-4a60-8f4e-0c6c71199e81 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.328412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 526/1076] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:13.329314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.332137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.332820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.333127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.333410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/d0f50a7555e64269ac35fd2b72a6cbd1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.335569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=844040a2fb60437a9188ccb5293d404b, role_id=d0f50a7555e64269ac35fd2b72a6cbd1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.337044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.358136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 551/1077] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:13 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/844040a2fb60437a9188ccb5293d404b/roles/d0f50a7555e64269ac35fd2b72a6cbd1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:13.440723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.459041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3d43a-9c7c-4478-b318-499783e3a984 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.501710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.502741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.503240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.503715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.504620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:13.505189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.506808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.518281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 527/1078] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:13.524025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.619336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.639330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989088-8555-4f77-ace7-c01b8184142b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.655032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.680995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.681681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.681907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.682173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] PATH_INFO: `/v3/users/9e280adb2a2e491e92e5e59f648e799d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.683299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] RBAC: Authorizing `identity:get_user(user_id=9e280adb2a2e491e92e5e59f648e799d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.684724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.686513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 552/1079] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/users/9e280adb2a2e491e92e5e59f648e799d => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:13.692085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.734958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc6b63-12d9-4f02-b1c4-5b3e5179f5b1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.770459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.771076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.771348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.771612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.792483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.797500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add49176-ce61-42bb-b71a-4838be29304f None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.837725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295662-558b-441b-b2ec-d423d21933da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.838535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295662-558b-441b-b2ec-d423d21933da None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.838814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295662-558b-441b-b2ec-d423d21933da None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.839410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75295662-558b-441b-b2ec-d423d21933da None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.840828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295662-558b-441b-b2ec-d423d21933da None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.843043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75295662-558b-441b-b2ec-d423d21933da None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.849832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 553/1080] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:13.850048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.850301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.850915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295662-558b-441b-b2ec-d423d21933da None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.911832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.967188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 528/1081] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:13.972320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75295662-558b-441b-b2ec-d423d21933da None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.973190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.013280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.014368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.014850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.015300 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340/users/9e280adb2a2e491e92e5e59f648e799d/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.017509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=31b6ce5375d84c4682351a53d1d5a340, user_id=9e280adb2a2e491e92e5e59f648e799d, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.019336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.029752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.042660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 554/1082] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:13 2022] PUT /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340/users/9e280adb2a2e491e92e5e59f648e799d/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:14.096044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e3400-fba5-4150-8ef4-19069cbda520 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.136767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.137442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.137721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.137989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.150304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.151935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e322525d-8bfe-468c-a766-1a9cc2db509d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:14.171120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:14.172216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:14.173218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:14.173218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:14.173218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:14.173218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 529/1083] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:14.180179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.299677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a448ac-fb2f-4c5b-aa51-b3251c4ffca2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.309931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.338891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.339675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.340050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.340415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.341230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:14.341629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.343098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a4985-1b60-4906-aac9-d6819824f8d1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.355594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 555/1084] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:15.535093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a1c1-2892-46f0-8e1a-dc2e2be2036a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.535665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a1c1-2892-46f0-8e1a-dc2e2be2036a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.535759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4439a1c1-2892-46f0-8e1a-dc2e2be2036a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.537141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 530/1085] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:15.543260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.543681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.543841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.562597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:15.563898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:15.638203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 556/1086] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:15.899636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a1c1-2892-46f0-8e1a-dc2e2be2036a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.007803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4439a1c1-2892-46f0-8e1a-dc2e2be2036a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.009351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.051245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.051869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.052142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.052404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.063810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-9b2215dd-52fe-4483-88fc-951106c9567b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:16.068539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.069519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:16.070410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 531/1087] 10.222.0.18 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:16.075809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.179571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.183853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752ec1c3-27af-4429-939f-bc76b9f5d56e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.216920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9fb91-f9ba-4a57-be3d-79e74d5388aa None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.217170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9fb91-f9ba-4a57-be3d-79e74d5388aa None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.217441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9fb91-f9ba-4a57-be3d-79e74d5388aa None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.219659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 532/1088] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:16.225623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.225885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.226137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.233254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.233886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.234164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.234493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.235125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.235474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.236837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.245408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 557/1089] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.248917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:16.250479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:16.251352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.324697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 533/1090] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:16.355066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61b45-51f4-4d7f-91e4-cbeb6c37ffe5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.388871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.389525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.389797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.390061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.400992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.402446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:16.417479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.418504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:16.419399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:16.419399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:16.419399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:16.419399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 558/1091] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:16.424803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.521003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.530405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7222b1f-57da-4e03-9e4a-2f8ec669156b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.574678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.575380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.575588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.575821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.576470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.576770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.578185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.588195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.592424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 534/1092] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.598383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.611275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.710064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c883a6-7c35-480b-91c0-7f8d42f9f5c2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.719747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b67bc1-e9a6-44ac-9bce-258dbb6befc1 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.754657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.754884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.755302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.755432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.755585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.755686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.755839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] PATH_INFO: `/v3/users/nova` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.755939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.764156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.765636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.767452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:16.772231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.778786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:16.779932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 559/1093] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:16.780796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.781990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:16.783219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 535/1094] 10.222.0.18 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:16.783947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.784031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.800489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.855810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.883888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74165fbd-8d0d-4df4-b6e7-42afe6b4229a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.884852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bbfed-a3cf-47ae-89ba-94ce90b9d7b5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.916726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.917720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.917990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.918255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.918629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.918977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.919258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.919359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.919535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.919805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.920466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.920687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.920790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.922145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.934419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 560/1095] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.939307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 536/1096] 10.222.0.18 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.940708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.944909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.012172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.041155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2352f0-6a88-4c62-8485-35cd0372b221 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.049917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a204bd-4126-4dc2-adf4-1a1ef930e0c3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.077266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.077894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.078172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.078430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] PATH_INFO: `/v3/users/9e280adb2a2e491e92e5e59f648e799d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.079624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=9e280adb2a2e491e92e5e59f648e799d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.081073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.082671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 561/1097] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/users/9e280adb2a2e491e92e5e59f648e799d => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:17.083548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.084151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.084426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.084692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.088355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.090986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.151860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.153377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.185680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483cf24a-aad1-4c1c-9026-a74b271c7bf7 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.219295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.220014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.220128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.220274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.220534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.252160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 537/1098] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:38:17.258103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.288643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.290046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.291845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.353711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f3a5b-3592-4f9d-9860-32d2ce0a0055 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.388509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.389196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.389471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.389720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 562/1099] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:17.389825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.395547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.401002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.402452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:17.416734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:17.418114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:17.419549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:17.419549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:17.419549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:17.419549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 538/1100] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:17.423893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.447750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.497909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3d434-3b8b-476d-a28c-02d391c84352 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.499427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.522715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eceae7-979e-4889-a593-b7322f684979 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.533554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.534181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.534582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.534852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.536120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.537656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.539374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 563/1101] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:17.545789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.557281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.557893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.558170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.558442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.559080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:17.559380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.560749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.572318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 539/1102] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:17.577950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.644821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49823ac0-197e-49e4-8821-dedba309915c None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.657980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.676727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11a20-f227-48de-b53e-5cee9558b51c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.680229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.680850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.681162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.681429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.682150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=31b6ce5375d84c4682351a53d1d5a340, user.id=9e280adb2a2e491e92e5e59f648e799d) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:17.682455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.683887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.691145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 564/1103] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/role_assignments?user.id=9e280adb2a2e491e92e5e59f648e799d&scope.project.id=31b6ce5375d84c4682351a53d1d5a340&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:17.714244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.714898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.715171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.715435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.723670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.725194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:17.737854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:17.738984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:17.739923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 540/1104] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:17.745311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.783780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.856141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455c1d5f-3011-4522-8cb8-cf1192f803ab None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.893762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.894387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.894663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.894952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.895653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:17.895952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.897449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79fbfa-eddc-43f4-8ae0-3b5b33e09c82 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.910134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 565/1105] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:17.916281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.018533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce04634-4c13-42f6-be9a-40ca758a3cea None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.055858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.056461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.056728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.057025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.057744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] RBAC: Adding query filter params (role.id=d0f50a7555e64269ac35fd2b72a6cbd1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=844040a2fb60437a9188ccb5293d404b) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:18.058041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.059451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1789f53-43e7-4948-bbd5-481207b84bad None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.065261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 541/1106] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/role_assignments?user.id=844040a2fb60437a9188ccb5293d404b&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=d0f50a7555e64269ac35fd2b72a6cbd1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:19.870952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f256-4aeb-466f-bda7-bc601b6c40c4 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.871521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f256-4aeb-466f-bda7-bc601b6c40c4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.871521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49f256-4aeb-466f-bda7-bc601b6c40c4 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.872918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 566/1107] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:19.878615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690038-e799-466d-b040-3533249f1926 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.879092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690038-e799-466d-b040-3533249f1926 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.879185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d690038-e799-466d-b040-3533249f1926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.892976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:19.898832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8d690038-e799-466d-b040-3533249f1926 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:19.900026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8d690038-e799-466d-b040-3533249f1926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:19.970264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 542/1108] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:20.234493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49f256-4aeb-466f-bda7-bc601b6c40c4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.342474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.359409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49f256-4aeb-466f-bda7-bc601b6c40c4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.400169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.400820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.401141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.401399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.402760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.404263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c70eb1-76e3-4926-89ca-1131e932e205 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.405996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 567/1109] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:20.412299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d690038-e799-466d-b040-3533249f1926 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.522269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d690038-e799-466d-b040-3533249f1926 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.557644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.558238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.558511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.558777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.559410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.560839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b037db8-afc7-4847-9c07-61da27fa1278 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.569887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.585637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 543/1110] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:20 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:22.770677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafc4d9-fbac-45de-ac56-5714628283b1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.771271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafc4d9-fbac-45de-ac56-5714628283b1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.771271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafc4d9-fbac-45de-ac56-5714628283b1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.772628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 568/1111] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:22.778178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.778635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.778728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.799382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:22.800569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:22.870004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 544/1112] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:23.138863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafc4d9-fbac-45de-ac56-5714628283b1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:23.248606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafc4d9-fbac-45de-ac56-5714628283b1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:23.283245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:23.283803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.284080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.284344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] PATH_INFO: `/v3/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.285857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] RBAC: Authorizing `identity:get_role(role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:23.287235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:23.288795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 569/1113] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:23 2022] GET /identity/v3/roles/b62430e4550a43989b6adfc24920894e => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:23.295638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:23.399557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabaf040-c179-4331-9572-48a7f893c6f4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:23.422404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:23.436226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:23.436854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.437170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.437437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] PATH_INFO: `/v3/users/32de8ac656914d969e395c41594ef057` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.450457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] RBAC: Authorizing `identity:get_user(user_id=32de8ac656914d969e395c41594ef057)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:23.451900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:23.453537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 545/1114] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:23 2022] GET /identity/v3/users/32de8ac656914d969e395c41594ef057 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:23.459691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:23.491007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:23.564565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4377f-2485-4c05-ba22-f2d703241f16 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:23.598019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:23.598628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.598905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.599174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.673513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:23.674973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:23.705170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:23.776330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 570/1115] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:23.782809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:23.884912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20ca05-6434-48d6-952e-238b8b2fdd5e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:23.920435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:23.921081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.921360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.921625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.922773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:23.924327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57eb55-17cf-480b-9a69-b5772a08bf3c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:23.925968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 546/1116] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:23 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:23.932517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.036827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ebc72-c5db-45a5-9217-ceb52dff48ff None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.066530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.073137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.073746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.074018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.074282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.075002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=31b6ce5375d84c4682351a53d1d5a340, user.id=32de8ac656914d969e395c41594ef057) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:24.075302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.076791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc18bdb-ab8f-4159-995f-0dec26446151 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:24.083245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 571/1117] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:23 2022] GET /identity/v3/role_assignments?user.id=32de8ac656914d969e395c41594ef057&scope.project.id=31b6ce5375d84c4682351a53d1d5a340&role.id=b62430e4550a43989b6adfc24920894e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:26.192793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b850d-218f-4864-8061-babac4fda883 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.193384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b850d-218f-4864-8061-babac4fda883 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.193384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b850d-218f-4864-8061-babac4fda883 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.194666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 547/1118] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:26.200094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.200511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.200637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.221173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:26.222405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:26.291658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 572/1119] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:26.565419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b850d-218f-4864-8061-babac4fda883 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.617735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.672238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b850d-218f-4864-8061-babac4fda883 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.711940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.713559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.713844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.714116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] PATH_INFO: `/v3/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.715553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] RBAC: Authorizing `identity:get_role(role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.716962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.718342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 548/1120] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/roles/b62430e4550a43989b6adfc24920894e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:26.725255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.832557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ff52b-74c4-46f2-9e8d-ad48b446dea6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.857925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.869793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.870434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.870705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.870971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] PATH_INFO: `/v3/users/32de8ac656914d969e395c41594ef057` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.872071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] RBAC: Authorizing `identity:get_user(user_id=32de8ac656914d969e395c41594ef057)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.873536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.875127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 573/1121] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/users/32de8ac656914d969e395c41594ef057 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:26.881406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.990317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4487a253-09dc-4a5b-9e41-2b12a454199a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:27.006830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:27.025853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:27.026489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:27.026732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:27.027123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:27.028159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:27.029630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b503a-e91d-4d0b-8156-cf4af1d63d96 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:27.031249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 549/1122] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:27.037439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:27.155738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cccea0-7d65-4316-88fe-db79f112a38a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:27.192165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:27.192804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:27.193134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:27.193402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340/users/32de8ac656914d969e395c41594ef057/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:27.195914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] RBAC: Authorizing `identity:create_grant(project_id=31b6ce5375d84c4682351a53d1d5a340, user_id=32de8ac656914d969e395c41594ef057, role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:27.197389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444be1a9-8551-440c-ac9a-09a6757f697d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:27.212335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 574/1123] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:27 2022] PUT /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340/users/32de8ac656914d969e395c41594ef057/roles/b62430e4550a43989b6adfc24920894e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:29.355874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4393-9933-4c2e-a338-09558f6f0df3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.356737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4393-9933-4c2e-a338-09558f6f0df3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.356737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e4393-9933-4c2e-a338-09558f6f0df3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.358860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 550/1124] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:29.365428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.365869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.365958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.384753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:29.385975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:29.443389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:29.473772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 575/1125] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:29.736710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4393-9933-4c2e-a338-09558f6f0df3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:29.872712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e4393-9933-4c2e-a338-09558f6f0df3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:29.908422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:29.909073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.909348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.909613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] PATH_INFO: `/v3/roles/b62430e4550a43989b6adfc24920894e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.911042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] RBAC: Authorizing `identity:get_role(role_id=b62430e4550a43989b6adfc24920894e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:29.912415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:29.913782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 551/1126] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:29 2022] GET /identity/v3/roles/b62430e4550a43989b6adfc24920894e => generated 223 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:29.920339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:30.027820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ca181-1725-4b4d-a9be-6688e3af780f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:30.066071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:30.066714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:30.066991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:30.067258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] PATH_INFO: `/v3/users/32de8ac656914d969e395c41594ef057` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:30.068443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] RBAC: Authorizing `identity:get_user(user_id=32de8ac656914d969e395c41594ef057)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:30.069971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:30.071583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 576/1127] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:29 2022] GET /identity/v3/users/32de8ac656914d969e395c41594ef057 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:30.077660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:30.146808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:30.198918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ddf1e-af5a-4dd1-bcc4-9817b90dbdd7 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:30.232845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:30.233490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:30.233766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:30.234035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:30.303484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:30.304939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:30.358366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:30.408527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 552/1128] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:30.415031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:30.466793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:30.518488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1515550b-662c-4d78-a2da-39ad915efa17 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:30.553970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:30.554510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:30.554786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:30.555052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] PATH_INFO: `/v3/projects/31b6ce5375d84c4682351a53d1d5a340` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:30.556205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] RBAC: Authorizing `identity:get_project(project_id=31b6ce5375d84c4682351a53d1d5a340)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:30.557664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e2611-e980-4989-9af4-ecb088c54586 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:30.559295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 577/1129] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:30 2022] GET /identity/v3/projects/31b6ce5375d84c4682351a53d1d5a340 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:30.565946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:30.667390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf484168-89b6-4977-ab16-b72fd42d9bc9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:30.706593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:30.707199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:30.707476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:30.707748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:30.708488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] RBAC: Adding query filter params (role.id=b62430e4550a43989b6adfc24920894e, scope.project.id=31b6ce5375d84c4682351a53d1d5a340, user.id=32de8ac656914d969e395c41594ef057) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:30.708795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:30.710270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9111ff-48e1-4225-8a4d-fd310c672363 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:30.716170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 553/1130] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:30 2022] GET /identity/v3/role_assignments?user.id=32de8ac656914d969e395c41594ef057&scope.project.id=31b6ce5375d84c4682351a53d1d5a340&role.id=b62430e4550a43989b6adfc24920894e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:32.881050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d8a-31e1-4547-92df-e33795392cca None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.881628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d8a-31e1-4547-92df-e33795392cca None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.881628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e200d8a-31e1-4547-92df-e33795392cca None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.882943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 578/1131] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:32.888613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.889037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.889184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.908918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:32.910300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:32.922171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:32.982496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 554/1132] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:33.246169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e200d8a-31e1-4547-92df-e33795392cca None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:33.351386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e200d8a-31e1-4547-92df-e33795392cca None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:33.369120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:33.386608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:33.387268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:33.387539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:33.387807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:33.388979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:33.390414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6b0dc-b1f6-4f1c-8f83-4c9fb73010c1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:33.392002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 579/1133] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:33.398110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:33.498954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993e5c3-50c3-45e3-b973-cb153e4f80e1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:33.533866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:33.534469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:33.534739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:33.535004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:33.535657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:33.537122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bb0df-fcad-4fc4-aa3e-01fc40a2a278 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:33.561255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:33.570707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 555/1134] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:33 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:35.724060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c2c0-659f-4081-a6ee-49e09d34f80c None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.724654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c2c0-659f-4081-a6ee-49e09d34f80c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.724654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295c2c0-659f-4081-a6ee-49e09d34f80c None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.725970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 580/1135] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:35.731371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.731819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.731913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.751388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:35.752718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:35.825660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 556/1136] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:36.088124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295c2c0-659f-4081-a6ee-49e09d34f80c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.191992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295c2c0-659f-4081-a6ee-49e09d34f80c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.227171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.227785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.228053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.228321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.229476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.230904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3215c9-c29a-4673-9f77-c293e796158b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.232416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 581/1137] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:36.238778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.337065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add67ef-6153-43e0-864f-20b0dfcaf5bc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.363501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.372068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.372671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.372981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.373252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.373899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.375338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be728428-47df-469a-83f5-28839b4d12a2 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.397127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 557/1138] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:36 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:38.563643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb2e-fb2c-4d55-b70c-a3ec891a92c9 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.564211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb2e-fb2c-4d55-b70c-a3ec891a92c9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.564211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcecb2e-fb2c-4d55-b70c-a3ec891a92c9 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.565609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 582/1139] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:38.571014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.571451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.571544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.590989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:38.592195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:38.662352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 558/1140] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:38.927500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcecb2e-fb2c-4d55-b70c-a3ec891a92c9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.985053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.034047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcecb2e-fb2c-4d55-b70c-a3ec891a92c9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.072178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.072805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.073109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.073373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.084193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:39.088936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:39.089957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:39.090890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 583/1141] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:39.096360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.197147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084b2a9-2c1a-4ea3-88e9-5b2ab11dbdb8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.211405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.232814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.233444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.233720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.233986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.234639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:39.234930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.236563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.244476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 559/1142] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:39.250769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.361360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebf034-86b2-486c-8139-5410d311b178 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.376260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.398992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.399626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.399898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.400161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] PATH_INFO: `/v3/users/e8c3b2d615444fecae2889c15289c222` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.413990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c3b2d615444fecae2889c15289c222)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.415447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.417087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 584/1143] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/users/e8c3b2d615444fecae2889c15289c222 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:39.423605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.526441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c92386-570a-419e-90f3-b5c821842ca9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.560486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.561116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.561395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.561658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.621843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.647142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.648588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.747133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 560/1144] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:39.753557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.858624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe1668-d810-4e9b-acd0-367bb192d045 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.893459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.894088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.894368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.894645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] PATH_INFO: `/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.895803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b04fd9e75047d98c816a71bcb63fdb)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.897266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c4f62-b80f-41d5-a283-c27840af1010 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.898886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 585/1145] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:39.905334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.982004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.005534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd615667-d178-452c-9de2-9147ff712249 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.041303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.041912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.042187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.042462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.043559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=d9b04fd9e75047d98c816a71bcb63fdb, user.id=e8c3b2d615444fecae2889c15289c222) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:40.043852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.045319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fae2b-8db0-4168-bdba-f986d403bf71 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.051200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 561/1146] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/role_assignments?user.id=e8c3b2d615444fecae2889c15289c222&scope.project.id=d9b04fd9e75047d98c816a71bcb63fdb&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:42.138897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23aad2-9f7b-4c93-bcf2-ae184401147b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.139481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23aad2-9f7b-4c93-bcf2-ae184401147b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.139481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23aad2-9f7b-4c93-bcf2-ae184401147b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.140833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 586/1147] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:42.146436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.146895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.147001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.165605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:42.166795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:42.237296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 562/1148] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:42.500220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23aad2-9f7b-4c93-bcf2-ae184401147b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:42.546127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:42.603975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23aad2-9f7b-4c93-bcf2-ae184401147b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.638129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.638755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.639031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.639327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.650125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:42.654960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:42.656111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:42.657369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 587/1149] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:42.668406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:42.741065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:42.772211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89326d-542e-4f41-8000-4a68a9eb0aae None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.808214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.808814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.809117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.809384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.810039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:42.810339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.811706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.819757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 563/1150] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:42.826012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:42.925489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:42.928848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9824ef-6899-46e4-8063-86e050a6268b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.963452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.964065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.964334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.964600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] PATH_INFO: `/v3/users/e8c3b2d615444fecae2889c15289c222` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.965788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c3b2d615444fecae2889c15289c222)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.967222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.968788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 588/1151] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/users/e8c3b2d615444fecae2889c15289c222 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:42.974860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:43.082515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f45d8-3892-4f65-bba9-ef0c67eb8ac5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:43.112703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:43.118688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:43.119314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:43.119603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:43.119870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] PATH_INFO: `/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:43.121067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b04fd9e75047d98c816a71bcb63fdb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:43.122509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c17ad5-b934-49f4-9215-2b06bdcbda47 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:43.124161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 564/1152] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:43.130622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:43.235823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dc0e5b-3681-4f74-a46d-5b5061cf90c6 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:43.271130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:43.271789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:43.272063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:43.272330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] PATH_INFO: `/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb/users/e8c3b2d615444fecae2889c15289c222/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:43.274342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9b04fd9e75047d98c816a71bcb63fdb, user_id=e8c3b2d615444fecae2889c15289c222, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:43.275771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5237c2ab-8762-425a-8d82-d479da62d8e7 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:43.290502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 589/1153] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:43 2022] PUT /identity/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb/users/e8c3b2d615444fecae2889c15289c222/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:45.401910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e948-23ff-4d8f-9226-74925ea361fc None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.403638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e948-23ff-4d8f-9226-74925ea361fc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.403924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4058e948-23ff-4d8f-9226-74925ea361fc None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.405301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 565/1154] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:45.410827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.411279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.411387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.430702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:45.431864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:45.454740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.501956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 590/1155] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:45.764911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e948-23ff-4d8f-9226-74925ea361fc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.869240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4058e948-23ff-4d8f-9226-74925ea361fc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.904901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.905549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.905829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.906096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.916921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:45.922033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:45.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:38:45.924547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 566/1156] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:45.929376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.032468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8a00f-c7ab-400a-aa6d-cae81fbd559e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.070226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.070830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.071100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.071362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.072019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:46.072316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.073704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.082071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 591/1157] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:46.088475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.156582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.197919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7e2a5-079e-4322-86eb-9afaebffbd7e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.234572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.235199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.235472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.235742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] PATH_INFO: `/v3/users/e8c3b2d615444fecae2889c15289c222` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.236970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c3b2d615444fecae2889c15289c222)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.238410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.239998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 567/1158] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:46 2022] GET /identity/v3/users/e8c3b2d615444fecae2889c15289c222 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:46.246949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.272332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.356941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91101034-17b3-497b-b6e6-1c4ccff80c24 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.392178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.392777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.393078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.393353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.460790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.462273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.482694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.562232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 592/1159] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:46.568528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.669762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113d902-73d1-43b9-8e5f-f78405fb7b97 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.696072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.704295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.705012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.705301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.705569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] PATH_INFO: `/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.706714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] RBAC: Authorizing `identity:get_project(project_id=d9b04fd9e75047d98c816a71bcb63fdb)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.708175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12da3441-3b5b-4822-9c8b-6f0e56f8c333 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.709810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 568/1160] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:46 2022] GET /identity/v3/projects/d9b04fd9e75047d98c816a71bcb63fdb => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:46.716206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.819020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c7568-20b7-49e2-a171-f07e050886e5 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.839801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.855298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.855926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.856200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.856455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.857202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=d9b04fd9e75047d98c816a71bcb63fdb, user.id=e8c3b2d615444fecae2889c15289c222) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:46.857501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.858911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4af34-955b-4c1a-bbb4-1785cab8571f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.865054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 593/1161] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:46 2022] GET /identity/v3/role_assignments?user.id=e8c3b2d615444fecae2889c15289c222&scope.project.id=d9b04fd9e75047d98c816a71bcb63fdb&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:49.013242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ddc5-8f31-40a8-bc6a-d0c922326839 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:49.013844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ddc5-8f31-40a8-bc6a-d0c922326839 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:49.013844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51ddc5-8f31-40a8-bc6a-d0c922326839 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:49.015114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 569/1162] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:49.020729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:49.021214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:49.021336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:49.040066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:49.041299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:49.110045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 594/1163] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:49.385080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51ddc5-8f31-40a8-bc6a-d0c922326839 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:49.498873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51ddc5-8f31-40a8-bc6a-d0c922326839 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:49.533782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:49.534385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:49.534653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:49.534900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:49.598436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:49.605314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:49.606766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:49.709075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 570/1164] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:49.715153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:49.814305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fa460-ce2d-4ae6-b3d6-f60f3a18f5b2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:49.828938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:49.850700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:49.851333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:49.851612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:49.851877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:49.858740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:49.860190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-15a15f2f-6ba2-481c-b0cc-da7cb93af6ca None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:49.871794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:49.872995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:38:49.873998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 595/1165] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:49.879241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:49.974938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:49.982568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cca9e-2151-46e8-acd6-9ca7567ab667 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:50.017811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:50.018481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:50.018671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:50.018942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:50.019588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:50.019885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:50.021277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e0230-f22a-4325-8ee7-a84d39fefefe None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:50.028837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 571/1166] 10.222.0.18 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:52.114044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d6f-6096-4959-b940-38a86e6d9739 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:52.114654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d6f-6096-4959-b940-38a86e6d9739 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:52.114654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d732d6f-6096-4959-b940-38a86e6d9739 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:52.115916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 596/1167] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:52.121319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c1ca-377e-438b-97f2-3128bbe2a17e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:52.121759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c1ca-377e-438b-97f2-3128bbe2a17e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:52.121851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320c1ca-377e-438b-97f2-3128bbe2a17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:52.140871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2320c1ca-377e-438b-97f2-3128bbe2a17e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:52.142096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2320c1ca-377e-438b-97f2-3128bbe2a17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:52.211244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 572/1168] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:52.479446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732d6f-6096-4959-b940-38a86e6d9739 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:52.592285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d732d6f-6096-4959-b940-38a86e6d9739 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:52.626033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:52.626635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:52.626910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:52.627172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:52.627828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:52.629214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724d845-138d-4ef7-8300-2cc27b85ad25 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:52.647526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 597/1169] 10.222.0.18 () {60 vars in 1138 bytes} [Tue Sep 13 19:38:52 2022] POST /identity/v3/domains => generated 267 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:54.801267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45db39-85a0-48fa-9ce7-73eb7af880b2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.801870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45db39-85a0-48fa-9ce7-73eb7af880b2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.801870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45db39-85a0-48fa-9ce7-73eb7af880b2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.803164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 573/1170] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:54.808809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.809314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.809406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.828610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:54.829890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:54.858346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:54.900352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 598/1171] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:55.166388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45db39-85a0-48fa-9ce7-73eb7af880b2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:55.260773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:55.272685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45db39-85a0-48fa-9ce7-73eb7af880b2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:55.308905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:55.309537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:55.309814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:55.310085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] PATH_INFO: `/v3/domains/d1f365b1ff2b4bc6afedbd9836ee53ad` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:55.321538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1f365b1ff2b4bc6afedbd9836ee53ad)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:55.323004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5bb5d-21a9-4cc2-b48d-80504bba8ae7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:55.324681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 574/1172] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 13 19:38:55 2022] GET /identity/v3/domains/d1f365b1ff2b4bc6afedbd9836ee53ad => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:55.331700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:55.440973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08bf8c-f1e1-4d8d-ab0f-8426c8b0f415 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:55.475993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:55.476607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:55.476917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:55.477183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:55.477830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:55.479264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7096d98-e299-4ef2-bdab-cda8ac298d33 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:55.507601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 599/1173] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:55 2022] POST /identity/v3/projects => generated 361 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:57.692247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885112f-a05c-468b-9ed4-f9364fb2c0bd None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.692857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885112f-a05c-468b-9ed4-f9364fb2c0bd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.692857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8885112f-a05c-468b-9ed4-f9364fb2c0bd None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.694151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 575/1174] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:57.700004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.700442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.700533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.719552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:57.720746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:57.732330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:57.799438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 600/1175] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:58.084172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885112f-a05c-468b-9ed4-f9364fb2c0bd None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:58.194295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8885112f-a05c-468b-9ed4-f9364fb2c0bd None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:58.229437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:58.230058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:58.230336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:58.230610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] PATH_INFO: `/v3/domains/d1f365b1ff2b4bc6afedbd9836ee53ad` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:58.231725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1f365b1ff2b4bc6afedbd9836ee53ad)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:58.233193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abda1706-9eb7-4d94-bd0c-00e498352cbe None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:58.234762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 576/1176] 10.222.0.18 () {56 vars in 1183 bytes} [Tue Sep 13 19:38:58 2022] GET /identity/v3/domains/d1f365b1ff2b4bc6afedbd9836ee53ad => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:58.240816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:58.357266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af923974-dcb4-4053-8037-d69f377bf046 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:58.391798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:58.392413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:58.392690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:58.393004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:58.393666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:58.395101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952c810-1934-4b25-bc86-9d3aefd644a3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:58.417676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 601/1177] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:58 2022] POST /identity/v3/users => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:00.612480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6b3a-56a1-4dc3-aab6-4593570f4dc2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:00.613162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6b3a-56a1-4dc3-aab6-4593570f4dc2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:00.613162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c6b3a-56a1-4dc3-aab6-4593570f4dc2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:00.614383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 577/1178] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:00.620021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:00.620449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:00.620561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:00.627467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:00.642316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:00.643532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:00.718557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 602/1179] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:00.989712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6b3a-56a1-4dc3-aab6-4593570f4dc2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:01.013952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:01.104908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c6b3a-56a1-4dc3-aab6-4593570f4dc2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.140714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.141365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.141640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.141904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.152838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:01.157655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:01.158708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:39:01.159757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 578/1180] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:01.165162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:01.268542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524af23-33bc-4cea-a933-3cd855ae014c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.298866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:01.304560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.305205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.305479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.305743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.306401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:01.306699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:01.308065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:01.316460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 603/1181] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:01.322805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:01.410949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:01.434515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4e76b-c143-4e7d-bfe2-0ca18196e529 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.469128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.469747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.470025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.470290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] PATH_INFO: `/v3/users/b6af6941523041c686f752de4ee7f2fa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.483771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] RBAC: Authorizing `identity:get_user(user_id=b6af6941523041c686f752de4ee7f2fa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:01.485249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:01.486916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 579/1182] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:01 2022] GET /identity/v3/users/b6af6941523041c686f752de4ee7f2fa => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:01.493176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:01.598581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd752b84-53da-47d6-b61c-ff65a52d4fe3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.634392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.635057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.635262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.635540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.699851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:01.708514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:01.710021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:01.814428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 604/1183] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:01.820783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:01.928514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03aecb-6b2e-4b2d-9753-c65e35a48f06 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.963932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.964557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.964809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.965114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] PATH_INFO: `/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.966268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] RBAC: Authorizing `identity:get_project(project_id=682a8c2aec6a43c5b78ef7521ab6d95b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:01.967699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cfbba-7453-4162-92b9-acf0b585e7fb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:01.969493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 580/1184] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:01 2022] GET /identity/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:01.976059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:02.075902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:02.083583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bb24b-ba87-4252-90ab-d8f3e4dc704d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:02.127713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:02.128326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:02.128600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:02.128865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:02.129632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=682a8c2aec6a43c5b78ef7521ab6d95b, user.id=b6af6941523041c686f752de4ee7f2fa) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:02.129942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:02.131381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb5e74-ef91-4943-9dea-5a664a5a7eae None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:02.137372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 605/1185] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:01 2022] GET /identity/v3/role_assignments?user.id=b6af6941523041c686f752de4ee7f2fa&scope.project.id=682a8c2aec6a43c5b78ef7521ab6d95b&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:04.254976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a90c-a114-4ce2-8da6-2deadf5fc0f6 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:04.255541 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a90c-a114-4ce2-8da6-2deadf5fc0f6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:04.255541 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21a90c-a114-4ce2-8da6-2deadf5fc0f6 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:04.256873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 581/1186] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:04.262222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:04.262661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:04.262749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:04.282159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:04.283254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:04.352817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 606/1187] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:04.617345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a90c-a114-4ce2-8da6-2deadf5fc0f6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:04.621836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:04.724406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21a90c-a114-4ce2-8da6-2deadf5fc0f6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:04.760761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:04.761439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:04.761712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:04.761978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:04.772740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:04.777519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:04.778712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:04.779913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:39:04.781092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 582/1188] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:04.788413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:04.887555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:04.900027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee09c9-edf1-4023-9681-df5f66b883d9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:04.935410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:04.936038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:04.936313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:04.936600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:04.937295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:04.937592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:04.938964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:04.947731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 607/1189] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:04.953975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:05.020585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:05.062965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efaa855-ab9b-4f6f-b330-fd7047b573bb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:05.098310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:05.098927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.099205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.099473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] PATH_INFO: `/v3/users/b6af6941523041c686f752de4ee7f2fa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.100651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] RBAC: Authorizing `identity:get_user(user_id=b6af6941523041c686f752de4ee7f2fa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:05.102165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:05.103744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 583/1190] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:04 2022] GET /identity/v3/users/b6af6941523041c686f752de4ee7f2fa => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:05.109833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:05.213391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86757665-947e-4363-83dd-07f47d54dfaf None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:05.247912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:05.248660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.249127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.249127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] PATH_INFO: `/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.250386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] RBAC: Authorizing `identity:get_project(project_id=682a8c2aec6a43c5b78ef7521ab6d95b)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:05.251814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b52735-c82d-4132-a9fc-1e8dcab29f74 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:05.253529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 608/1191] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:05 2022] GET /identity/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:05.259817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:05.360444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aed7f9-ac9e-4dea-b6c5-a84ae53d1d10 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:05.391020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:05.396392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:05.397081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:05.397359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:05.397630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] PATH_INFO: `/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b/users/b6af6941523041c686f752de4ee7f2fa/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:05.399545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=682a8c2aec6a43c5b78ef7521ab6d95b, user_id=b6af6941523041c686f752de4ee7f2fa, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:05.401000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb1021-e916-44d2-a3d7-9170525952e7 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:05.414555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 584/1192] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:39:05 2022] PUT /identity/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b/users/b6af6941523041c686f752de4ee7f2fa/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:39:07.528432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7187a8-7e3c-449c-994d-ecb2c0bb9676 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:07.529037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7187a8-7e3c-449c-994d-ecb2c0bb9676 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:07.529037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7187a8-7e3c-449c-994d-ecb2c0bb9676 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:07.530369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 609/1193] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:07.535877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:07.536326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:07.536418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:07.556059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:07.557279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:07.625119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 585/1194] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:07.891634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7187a8-7e3c-449c-994d-ecb2c0bb9676 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:07.912273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:07.998910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7187a8-7e3c-449c-994d-ecb2c0bb9676 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.034569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.035208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.035481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.035746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.046646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:08.051310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:08.052401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:08.053553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:39:08.057042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 610/1195] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:08.058509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:08.158958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303ebaf-b92d-4508-833e-57985970dde2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.183712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:08.196979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.197591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.197872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.198145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.198814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:08.199118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:08.200496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:08.208528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 586/1196] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:08.215047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:08.294740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:08.317410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307a2df-4bbe-47dc-9960-ce3e3dbe812a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.350833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.351455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.351727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.352006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] PATH_INFO: `/v3/users/b6af6941523041c686f752de4ee7f2fa` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.353175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] RBAC: Authorizing `identity:get_user(user_id=b6af6941523041c686f752de4ee7f2fa)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:08.354602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:08.356153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 611/1197] 10.222.0.18 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:08 2022] GET /identity/v3/users/b6af6941523041c686f752de4ee7f2fa => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:39:08.362197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:08.464477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce742372-64ec-4cba-ba4d-c39092a3dcad None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.500243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.500855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.501171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.501441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.554343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:08.571210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:08.572660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:08.673981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 587/1198] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:08.680631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:08.786519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b69b1-9333-4f67-a718-aab6b4e24451 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.821899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.822534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.822812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.823084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] PATH_INFO: `/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.824249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] RBAC: Authorizing `identity:get_project(project_id=682a8c2aec6a43c5b78ef7521ab6d95b)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:08.825724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17119fc-f34a-4c9e-aec2-37653a2a30ec None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:08.827363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 612/1199] 10.222.0.18 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:08 2022] GET /identity/v3/projects/682a8c2aec6a43c5b78ef7521ab6d95b => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:08.834065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:08.922080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:08.938766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e9afa-48de-4662-aa4d-52381de5021a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:08.973398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:08.973990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:08.974248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:08.974517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:08.975220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=682a8c2aec6a43c5b78ef7521ab6d95b, user.id=b6af6941523041c686f752de4ee7f2fa) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:08.975521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:08.976968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ab33b-2562-4548-8b40-2368c9f79ce6 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:08.982583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 588/1200] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:08 2022] GET /identity/v3/role_assignments?user.id=b6af6941523041c686f752de4ee7f2fa&scope.project.id=682a8c2aec6a43c5b78ef7521ab6d95b&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:18.817081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:18.980088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=26 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:05.064925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e22b-ca58-4852-a716-4cfc73878fb6 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:05.065521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e22b-ca58-4852-a716-4cfc73878fb6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:05.065521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e34e22b-ca58-4852-a716-4cfc73878fb6 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:05.067107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 613/1201] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:05.073022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:05.073555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:05.073555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:05.092377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:05.093590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:05.177308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 589/1202] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:05.446622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34e22b-ca58-4852-a716-4cfc73878fb6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:05.569778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e34e22b-ca58-4852-a716-4cfc73878fb6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:05.616347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:05.617145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:05.617430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:05.617886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:05.630432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:05.632317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:05.650003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:05.651307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:05.652289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:05.652289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:05.652289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:05.652289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 614/1203] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:05.658032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:05.788865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def47532-8235-42f5-b88d-ee4c13c5b6dd None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:05.822721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:05.829441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:05.830064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:05.830389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:05.830701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:05.831355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:05.831880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:05.833555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679159c-bbc9-4f09-8a28-71a7208b6d7d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:05.850878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 590/1204] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:05.852631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:05.928097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:05.993096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d52-830e-4cf9-a825-36bca66f17cf None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:06.054052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:06.054709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:06.055022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:06.055292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:06.055990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:06.057606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209d73a-08d2-422a-94c6-142a5f4cb042 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:06.088289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 615/1205] 10.222.0.18 () {60 vars in 1132 bytes} [Tue Sep 13 19:40:05 2022] POST /identity/v3/users => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:40:08.530261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bfd12-c0d0-4fa1-8af2-388d6e0b6c29 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:08.530891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bfd12-c0d0-4fa1-8af2-388d6e0b6c29 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:08.530891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bfd12-c0d0-4fa1-8af2-388d6e0b6c29 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:08.532124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 591/1206] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:40:08.538130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:08.538669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:08.538822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:08.558564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:08.559884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:08.647340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 616/1207] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:40:08.932244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bfd12-c0d0-4fa1-8af2-388d6e0b6c29 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.059289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bfd12-c0d0-4fa1-8af2-388d6e0b6c29 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:09.099626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:09.100268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:09.100535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:09.100803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:09.114282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:09.119088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:09.121618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:09.122481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 592/1208] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:09.126588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.170202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:09.241498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e449e3-04e5-4ecb-82cc-9c199106a54b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:09.280924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:09.281531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:09.281803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:09.282065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:09.282734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:09.283034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:09.284694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:09.294903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 617/1209] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:09.300877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.358801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:09.411838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821cd11-ec3b-41d4-8c5b-ad6e4f21a411 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:09.447752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:09.448397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:09.448675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:09.448986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:09.460080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:09.461574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:09.476738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:09.478700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:09.480320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:09.480320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:09.480320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:09.480320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 593/1210] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:09.487725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.612690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c5d5d-0031-49f5-af4e-4d3b6914fb5b None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:09.613392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:09.658256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:09.658916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:09.659213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:09.659551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:09.660227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:09.660596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:09.662122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:09.676534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 618/1211] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:09.683051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.821794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39273a2-211b-4067-9794-ae73c46e8944 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:09.823407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:09.860272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:09.861522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:09.861522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:09.861769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:09.870429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:09.871808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:09.883591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:09.885577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:09.886561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 594/1212] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:09.890787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:09.995753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6b1500-3e80-4809-a166-6273d82cb6b4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:10.032004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:10.032633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:10.032982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:10.033263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:10.034037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:10.034372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:10.036023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:10.054917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 619/1213] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:10.060599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:10.179854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c805478-7885-4759-8f66-7592ed4922f4 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:10.232796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:10.233699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:10.234175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:10.234620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:10.266976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:10.312032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:10.313697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:10.430380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 595/1214] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:40:10.438043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:10.462282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:10.551584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85ace0-3959-4d64-bd46-6165221a5ea2 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:10.594411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:10.595138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:10.595441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:10.595731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:10.607334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:10.608964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:10.626197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:10.627735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:10.628626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:10.628626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:10.628626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:10.628626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 620/1215] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:10.633654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:10.702901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:10.750927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8de51-c1cd-4b94-8db5-6164c1c7c403 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:10.789837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:10.790494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:10.790803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:10.791087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:10.791751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:10.792054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:10.793652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:10.808731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 596/1216] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:10.814616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:10.922907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:10.957709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0010df5-a546-46f6-93d8-d2ed00d18a33 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:10.993263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:10.993901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:10.994183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:10.994447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:11.001331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:11.002781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:11.015097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:11.016287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:11.017265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 621/1217] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:11.022707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:11.132029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d69189-0b9e-4908-9364-d04f332da2f5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:11.156664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:11.169272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:11.169896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:11.170221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:11.170466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:11.171576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:11.171900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:11.173504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a07a86-e302-405f-8db3-dc2147962997 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:11.193164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 597/1218] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:11.194551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:11.304297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e716c6-042a-4f5f-9a60-33970d87b16d None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:11.335846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:11.342586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:11.343203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:11.343480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:11.343756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:11.344471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=d4895ca8a2854fb38ce08ce74b628671) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:11.344777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:11.346254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eead4b-b65c-4f78-81d3-463ad0a4555a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:11.352265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 622/1219] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:11 2022] GET /identity/v3/role_assignments?user.id=d4895ca8a2854fb38ce08ce74b628671&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:13.518640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b3d0-5b0b-487a-aace-285523a694de None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:13.519329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b3d0-5b0b-487a-aace-285523a694de None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:13.519329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b3d0-5b0b-487a-aace-285523a694de None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:13.520689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 598/1220] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:13.526189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:13.526738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:13.526738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:13.545660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:13.547271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:13.622401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 623/1221] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:13.891733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b3d0-5b0b-487a-aace-285523a694de None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.007778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b3d0-5b0b-487a-aace-285523a694de None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:14.061940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:14.062684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:14.062965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:14.063241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:14.076494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:14.081853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:14.083058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:14.084035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 599/1222] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:14.089833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.203350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055edb4-d3fb-496f-8f55-0cd47b6799a0 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:14.229470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:14.242410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:14.243020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:14.243376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:14.243613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:14.244245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] RBAC: Adding query filter params (name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:14.244542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:14.245990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:14.255572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 624/1223] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:14.261879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.309741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:14.371889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91366e44-afbf-4cb1-9045-f982055121b3 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:14.416684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:14.417694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:14.418002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:14.418347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:14.430831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:14.432391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:14.449633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:14.450732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:14.451636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:14.451636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:14.451636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:14.451636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 600/1224] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:14.462377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.587933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9465f-f9d8-45d6-9a4e-1ca11dc66ee3 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:14.637733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:14.638486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:14.638828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:14.639159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:14.639957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:14.640319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:14.641973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:14.658000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 625/1225] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:14.664137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.782515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:14.791643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad2e12-99b9-4f5f-9e56-2105dd80c09e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:14.834393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:14.835358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:14.835798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:14.836060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:14.844335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:14.845930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:14.858936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:14.860496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:14.862392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 601/1226] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:14.866601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:14.934773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:14.999951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047f9f8-5b1d-4b14-b6b0-6aee7fc01634 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:15.042317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:15.042879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:15.043204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:15.043476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:15.044212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:15.044515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:15.046341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:15.064776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 626/1227] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:15.071277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:15.192792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca919d-698a-40be-b3d4-3a28d6bf2b07 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:15.239279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:15.240032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:15.240333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:15.240624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:15.253700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:15.255584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:15.278407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:15.279623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:15.280774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:15.280774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:15.280774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:15.287478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 602/1228] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:15.287703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:15.411713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43e0be-bf8a-467b-a026-bb6e72c63f66 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:15.414492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:15.457811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:15.457811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:15.457811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:15.457811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:15.458334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:15.458334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:15.458492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:15.471929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 627/1229] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:15.478080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:15.488445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:15.588586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40eeeef-4aa2-4f52-aeb4-0b69d9f44c46 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:15.632539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:15.633206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:15.633485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:15.633745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:15.640954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:15.642417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:15.654679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:15.655875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:15.656782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 603/1230] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:15.662433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:15.784852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f694070-7540-420d-a808-cff1b9b27ae9 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:15.826302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:15.826932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:15.827209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:15.827536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:15.828173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:15.828506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:15.829956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad29a68-c9f5-4df8-8143-038b7d54728e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:15.842921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 628/1231] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:15.850319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:15.922027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:15.961473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43ace8-83f6-4253-ab5f-f32123780ffe None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:16.002200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:16.002911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:16.003193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:16.003526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/d4895ca8a2854fb38ce08ce74b628671/roles/ba59a76cd3734dd49c13e485e881e8b1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:16.019754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=d4895ca8a2854fb38ce08ce74b628671, role_id=ba59a76cd3734dd49c13e485e881e8b1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:16.021263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929da4a3-13a8-4029-8339-042a45580a9d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:16.036619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 604/1232] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:40:15 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/d4895ca8a2854fb38ce08ce74b628671/roles/ba59a76cd3734dd49c13e485e881e8b1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:40:18.171966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849722e3-b52e-476a-8564-cb8f205fb2da None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:18.172550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849722e3-b52e-476a-8564-cb8f205fb2da None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:18.172550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849722e3-b52e-476a-8564-cb8f205fb2da None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:18.174013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 629/1233] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:18.179460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:18.179866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:18.180006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:18.199298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:18.200638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:18.271334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 605/1234] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:18.533816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849722e3-b52e-476a-8564-cb8f205fb2da None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:18.562491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:18.664951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849722e3-b52e-476a-8564-cb8f205fb2da None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:18.701246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:18.701891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:18.702221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:18.702537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:18.714891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:18.719795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:18.720994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:18.721869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 630/1235] 10.222.0.18 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:18.727237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:18.774979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:18.848601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd381aab-68ac-4bb9-985b-306ba37c3928 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:18.890104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:18.890721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:18.890994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:18.891259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:18.892035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] RBAC: Adding query filter params (name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:18.892357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:18.893851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:18.902715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 606/1236] 10.222.0.18 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:18.908941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:19.004408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:19.022219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28730946-e79f-43aa-9e4a-2fa9e90ba2b1 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:19.062744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:19.063496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:19.063772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:19.064042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:19.074952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:19.076406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:19.092186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:19.094199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:19.096049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:19.096049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:19.096049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:19.096049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 631/1237] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:19.100544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:19.230838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bcf39-17ea-46c0-8893-2770ffaa7051 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:19.240697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:19.267810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:19.268423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:19.268697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:19.268998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:19.269896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:19.270234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:19.271673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:19.285556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 607/1238] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:19.291677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:19.405083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3018cb7-b99a-44a3-afd3-61d0b66e1773 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:19.446422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:19.447056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:19.447327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:19.447590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:19.450406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:19.457753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:19.459587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:19.473237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:19.474246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:19.475183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 632/1239] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:19.483556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:19.641969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564f045-6bb9-446e-be6f-eae6f14be3a9 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:19.678042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:19.678572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:19.678850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:19.679118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:19.679843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:19.680145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:19.681627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:19.701890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 608/1240] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:19.707914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:19.839570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e81bf6d-c024-4965-82ae-7023a178dbe4 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:19.874284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:19.874918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:19.875192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:19.875457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:19.946725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:19.954422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:19.955888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:20.069108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 633/1241] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:40:20.075332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:20.104916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:20.206742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7246bd7b-2583-40e6-906b-fce6af593508 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:20.249033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:20.250061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:20.250516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:20.250839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:20.265023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:20.265730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:20.282027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:20.289781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:20.290863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:20.290863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:20.290863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:20.290863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 609/1242] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:20.291124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:20.395769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:20.402696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a66ba-7f00-438a-9774-babae56c2703 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:20.438829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:20.439448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:20.439722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:20.439991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:20.440628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:20.441050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:20.442419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:20.454485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 634/1243] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:20.461319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:20.559229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:20.601597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e59e6-f5cc-4dc0-89da-530e100bb92b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:20.647503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:20.648605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:20.649268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:20.649770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:20.660355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:20.662187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:20.674733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:20.675902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:20.677007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 610/1244] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:20.682580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:20.798196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d5ff4-88ca-4739-92a4-2dfd5ba065f6 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:20.834880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:20.835496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:20.835773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:20.836040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:20.836728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:20.837082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:20.838505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc357bf-531a-4710-963c-8c515cf8f2e0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:20.851366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 635/1245] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:20.858434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:20.977867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f434d-3973-4ac5-88c9-d56f8af9a864 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:21.019652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:21.020272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:21.020737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:21.020839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:21.021578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] RBAC: Adding query filter params (role.id=ba59a76cd3734dd49c13e485e881e8b1, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=d4895ca8a2854fb38ce08ce74b628671) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:21.021859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:21.023324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237490f5-cdd9-48b8-bcac-bf856ba627ac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:21.030425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 611/1246] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:20 2022] GET /identity/v3/role_assignments?user.id=d4895ca8a2854fb38ce08ce74b628671&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=ba59a76cd3734dd49c13e485e881e8b1 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:23.386319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc16b-6b10-4870-ab70-5a30c4930d09 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:23.386966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc16b-6b10-4870-ab70-5a30c4930d09 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:23.386966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edc16b-6b10-4870-ab70-5a30c4930d09 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:23.388421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 636/1247] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:23.394453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:23.394914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:23.395031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:23.401955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:23.416246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:23.417471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:23.495085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 612/1248] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:23.784344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edc16b-6b10-4870-ab70-5a30c4930d09 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:23.805451 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:23.903530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edc16b-6b10-4870-ab70-5a30c4930d09 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:23.942000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:23.942646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:23.942926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:23.943199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:23.953607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:23.958508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:23.959730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:23.960830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:23.964814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 637/1249] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:23.965872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:24.088225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3807547-c7b4-403c-b4a8-ec5c98e01602 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:24.123577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:24.131320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:24.132053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:24.132368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:24.132698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:24.133489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:24.133791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:24.135216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:24.144989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 613/1250] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:24.151672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:24.244223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:24.269626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e4e0d-77f6-48af-96cc-8d0dd6b25714 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:24.311967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:24.312621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:24.312965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:24.313243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:24.326865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:24.328412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:24.346609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:24.348020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:24.349205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:24.349205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:24.349205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:24.349205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 638/1251] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:24.358752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:24.475626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e542870-5521-42f2-aca9-c2d7f69b5785 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:24.512791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:24.513470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:24.513828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:24.514102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:24.514915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:24.515220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:24.516694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:24.531717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 614/1252] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:24.538009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:24.657558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a206f-bdac-4a50-93bd-876437720c38 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:24.684565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:24.693222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:24.693865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:24.694137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:24.694403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:24.702599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:24.704052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:24.716944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:24.718402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:24.719583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 639/1253] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:24.726478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:24.790046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:24.843719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2da1e7-b900-4cb7-831f-50fae3159a8e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:24.880790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:24.881389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:24.881655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:24.881962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:24.882652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:24.882970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:24.884411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:24.902829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 615/1254] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:24.909121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:25.031871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314b814-41af-486b-89d3-2e079893db84 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:25.068577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:25.069249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:25.069552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:25.069795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:25.130162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:25.146435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:25.147905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:25.258598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 640/1255] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:40:25.265717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:25.389926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3f6c3-b022-49f5-8066-d3870b54da47 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:25.405004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:25.433023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:25.433745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:25.434029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:25.434358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:25.446828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:25.448533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:25.465047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:25.466343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:25.467657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:25.467657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:25.467657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:25.467657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 616/1256] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:25.472466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:25.579526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:25.595257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec7a812-ed45-4ab8-ac5d-2e95bab936c8 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:25.637423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:25.638070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:25.638345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:25.638619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:25.639273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:25.639588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:25.641144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:25.654256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 641/1257] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:25.660818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:25.769083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96db079c-8d8a-4777-9279-c8238f0092de None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:25.808332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:25.809005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:25.809280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:25.809556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:25.817511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:25.818983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:25.831411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:25.832482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:25.838664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 617/1258] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:25.840773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:25.950077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1b0c5-23ea-4cf2-93f5-ded38de7d04a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:25.983850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:25.984463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:25.984738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:25.985040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:25.985730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:25.986028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:25.987521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c2570-48f8-4c71-8e8a-e5d25571279f None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:26.000224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 642/1259] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:26.006784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:26.036092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:26.113829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15243f-8032-4b57-8e32-e142a32f6a03 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:26.149724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:26.150492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:26.150865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:26.151234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:26.152063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=d4895ca8a2854fb38ce08ce74b628671) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:26.152369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:26.153878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd30fd0-c201-4e97-a85e-043f3b9027ac None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:26.159784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 618/1260] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:26 2022] GET /identity/v3/role_assignments?user.id=d4895ca8a2854fb38ce08ce74b628671&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:28.279112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599b566-c4ca-477b-be0f-09b53cc05547 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:28.279646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599b566-c4ca-477b-be0f-09b53cc05547 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:28.279646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e599b566-c4ca-477b-be0f-09b53cc05547 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:28.280861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 643/1261] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:28.286942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:28.287413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:28.287529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:28.309728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:28.310922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:28.382425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 619/1262] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:28.645227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599b566-c4ca-477b-be0f-09b53cc05547 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:28.658050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:28.757393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e599b566-c4ca-477b-be0f-09b53cc05547 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:28.792671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:28.793340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:28.793619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:28.794243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:28.805195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:28.809916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:28.810966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:28.811973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 644/1263] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:28.817556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:28.837140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:28.921015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf9ebd-9f57-4f00-aa4e-85fa032251dc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:28.956778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:28.957418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:28.957697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:28.957965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:28.958621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:28.958923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:28.960292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:28.968773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 620/1264] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:28.974857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.050990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:29.086579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d863bf-dff4-4bc5-a8a7-94dfa63d6d5a None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:29.120409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:29.121059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:29.121333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:29.121599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:29.132373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:29.133874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:29.148131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:29.149354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:29.150324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:29.150324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:29.150324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:29.150324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 645/1265] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:29.155511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.235312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:29.256831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d79f9-ae14-49ce-9a4b-e5f185bc941a None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:29.302293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:29.303020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:29.303124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:29.303411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:29.304051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:29.304349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:29.305754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:29.318353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 621/1266] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:29.324340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.427631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d02a14-ad31-4455-b8a0-d0f60da0c42e None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:29.447710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:29.463012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:29.463631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:29.463902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:29.464168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] PATH_INFO: `/v3/users/placement` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:29.471878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:29.473360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:29.484345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:29.485485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:29.486729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 646/1267] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:29.492277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.615534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403939b5-1f0f-423f-bd7c-12b37918aebb None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:29.652568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:29.653216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:29.653491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:29.653755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:29.654480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:29.655422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:29.656213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:29.665940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:29.677620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 622/1268] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:29.683732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.797264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a93950-14c1-4d2f-ad1c-f545978354cc None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:29.836935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:29.837582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:29.837852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:29.838110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:29.849291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:29.850743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:29.865549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:29.867310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:29.869065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:29.869065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:29.869065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:29.869065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 647/1269] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:29.872679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:29.980952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19d07b-c6a2-494a-aaed-da1bf6934f37 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:30.018268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:30.018867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:30.019118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:30.019387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:30.020019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:30.020320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:30.021730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:30.034026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 623/1270] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:30.040021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:30.082667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:30.165040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe98ee-84bf-467b-b596-343cada18cfa None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:30.200085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:30.200718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:30.201028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:30.201294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:30.208513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:30.209985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:30.221402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:30.222403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:30.223756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 648/1271] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:30.228627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:30.300141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:30.346915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96930d8b-4689-4bf0-840e-06d845653a7c None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:30.382761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:30.383359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:30.383638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:30.383901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:30.384574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:30.384870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:30.386329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8237a-b434-45d2-b647-cef2ebc9cede None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:30.399233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 624/1272] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:30.405776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:30.511673 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:30.522576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec929d-9300-47fc-81f0-88f25f3e1f01 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:30.558732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:30.559509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:30.559789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:30.560062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] PATH_INFO: `/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/d4895ca8a2854fb38ce08ce74b628671/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:30.562545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] RBAC: Authorizing `identity:create_grant(project_id=12814ce72efb432b9bfaa111d095abb9, user_id=d4895ca8a2854fb38ce08ce74b628671, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:30.564638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17a21f7-ffe5-4669-984a-3a544577eed0 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:30.582947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 649/1273] 10.222.0.18 () {58 vars in 1439 bytes} [Tue Sep 13 19:40:30 2022] PUT /identity/v3/projects/12814ce72efb432b9bfaa111d095abb9/users/d4895ca8a2854fb38ce08ce74b628671/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:40:32.728453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988fcac-b459-4c12-afe0-e72a453bb12b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:32.729084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988fcac-b459-4c12-afe0-e72a453bb12b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:32.729084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2988fcac-b459-4c12-afe0-e72a453bb12b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:32.730358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 625/1274] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:32.735851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:32.736398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:32.736509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:32.759588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:32.760788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:32.833259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 650/1275] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:33.115731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988fcac-b459-4c12-afe0-e72a453bb12b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.221352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2988fcac-b459-4c12-afe0-e72a453bb12b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:33.238525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:33.256376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:33.257022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:33.257270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:33.257559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:33.267613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:33.272307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:33.273462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:33.274329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 626/1276] 10.222.0.18 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:33.280079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.363111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:33.382477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfed547b-c3e2-4cbb-b9af-65f7410a0f1e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:33.416757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:33.417386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:33.417659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:33.417927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:33.418933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:33.419232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:33.420614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:33.429035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 651/1277] 10.222.0.18 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:33.435456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.541630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd30831-8082-4db0-95a8-3a26990327b6 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:33.579041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:33.579890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:33.580194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:33.580553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:33.593015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:33.594536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:33.612068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:33.613212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:33.614121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:33.614121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:33.614121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:33.614121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 627/1278] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:33.619485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.726205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b802369-79b5-4a92-8818-342b991ef350 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:33.760379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:33.761104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:33.761285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:33.761551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:33.762180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:33.762479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:33.763822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:33.775791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 652/1279] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:33.781979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.814419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:33.885548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd20c93-0e60-49a5-a10e-165b48279c6e None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:33.920217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:33.920840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:33.921138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:33.921402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] PATH_INFO: `/v3/users/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:33.929073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:33.930506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:33.942171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:33.943304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:33.944310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 628/1280] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:33.949507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:33.953442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:34.052628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d6a6fb-1f32-47c4-a876-a9bb8ddd4b82 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:34.088060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:34.088582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.088853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.089124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.089873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:34.090177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:34.091593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:34.109240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 653/1281] 10.222.0.18 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:34.115526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:34.212159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:34.222901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67b2a9-4d71-4f8b-9fdf-fb4f01ebe78d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:34.259586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:34.260184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.260456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.260720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.329808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:34.331260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:34.420200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:34.432260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 629/1282] 10.222.0.18 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:40:34.438425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:34.505944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:34.547906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387042d-96ac-425d-b3a0-378d5cb01136 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:34.585460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:34.586097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.586374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.586640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.597356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:34.598838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:34.613653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:34.614797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:34.615957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:34.615957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:34.615957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:34.615957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 654/1283] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:34.620776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:34.720071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80371a-66cf-4c8f-bbb6-6bca108a197b None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:34.755435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:34.756010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.756302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.756574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.757239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:34.757535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:34.758914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:34.770550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 630/1284] 10.222.0.18 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:34.776507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:34.893649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95402b0-00f5-4e64-89d7-0e4f0a92ad82 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:34.908291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:34.930377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:34.931009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.931266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.931532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.938260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:34.939750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:34.950659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:34.951620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:34.952703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: ERROR keystone.server.flask.application Sep 13 19:40:34.956790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 655/1285] 10.222.0.18 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:34.957962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.007945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:35.060763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc25208-5623-483a-91c9-a677725b7f1d None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.096697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.097319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.097585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.097849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.098527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:35.098822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.100243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892f3d6-9e27-4994-8323-6a89178b2ec3 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.112317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 631/1286] 10.222.0.18 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:35.118597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.240603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9d796-7ab1-44f5-b587-a16fe17c26e1 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.277137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.277654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.277927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.278191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.278902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] RBAC: Adding query filter params (role.id=8b79e1e620da45a9bc66fc22b41b2a58, scope.project.id=12814ce72efb432b9bfaa111d095abb9, user.id=d4895ca8a2854fb38ce08ce74b628671) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:35.279204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.280632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491c039-4fa1-4300-8140-2c6dd219210e None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.286656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 656/1287] 10.222.0.18 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/role_assignments?user.id=d4895ca8a2854fb38ce08ce74b628671&scope.project.id=12814ce72efb432b9bfaa111d095abb9&role.id=8b79e1e620da45a9bc66fc22b41b2a58 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:37.418389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae50bc-3437-46cd-be56-a590eb64cb9e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.418952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae50bc-3437-46cd-be56-a590eb64cb9e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.418952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eae50bc-3437-46cd-be56-a590eb64cb9e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.420269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 632/1288] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:37.425593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.426042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.426143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.444462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:37.445655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:37.475535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:37.516312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 657/1289] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:37.795555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae50bc-3437-46cd-be56-a590eb64cb9e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:37.899239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eae50bc-3437-46cd-be56-a590eb64cb9e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:37.910121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:37.935388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:37.936015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.936292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.936558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.937167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:37.938535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:37.947468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:37.948673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:37.949600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 633/1290] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:40:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:37.954877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:38.058326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c347db3-d495-4d0f-a405-30e39b041545 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:38.092653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:38.093288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:38.093568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:38.093836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:38.094457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] RBAC: Adding query filter params (name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:38.094756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:38.096143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8c1cb-6bf0-45ef-9334-0af6f68e272a None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:38.100960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 658/1291] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 19:40:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:38.107046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:38.209798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e8197-e2f9-497b-8d5a-9f56e3114655 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:38.245982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:38.246582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:38.246856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:38.247120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] PATH_INFO: `/v3/services` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:38.247747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:38.248042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:38.249441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b17df1-5d2a-4b6e-a846-3db0bd0d5706 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:38.254584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 634/1292] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 19:40:38 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:40.374066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e974-7f61-48a2-b89e-6aa48499dcbb None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.374664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e974-7f61-48a2-b89e-6aa48499dcbb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.374664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49e974-7f61-48a2-b89e-6aa48499dcbb None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.375963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 659/1293] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:40.381326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67a2f-485f-45d6-85a5-32b3ac401ddb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.381796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67a2f-485f-45d6-85a5-32b3ac401ddb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.381888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc67a2f-485f-45d6-85a5-32b3ac401ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.401288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ecc67a2f-485f-45d6-85a5-32b3ac401ddb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:40.402483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ecc67a2f-485f-45d6-85a5-32b3ac401ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:40.434088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.473382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 635/1294] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:40.737799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e974-7f61-48a2-b89e-6aa48499dcbb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.770846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.847760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49e974-7f61-48a2-b89e-6aa48499dcbb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.883633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.884237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.884512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.884779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.885454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.886819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be8158-c2da-4e79-89fe-0319a86d7a32 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:40.903295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 660/1295] 10.222.0.18 () {60 vars in 1141 bytes} [Tue Sep 13 19:40:40 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:40:43.018874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99ac-50ac-478a-a63e-e3079fa036f3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.019552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99ac-50ac-478a-a63e-e3079fa036f3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.019552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8a99ac-50ac-478a-a63e-e3079fa036f3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.020767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 636/1296] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:43.026656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.026656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.027132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.046384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:43.047560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:43.121585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 661/1297] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:43.386922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a99ac-50ac-478a-a63e-e3079fa036f3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:43.492271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8a99ac-50ac-478a-a63e-e3079fa036f3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:43.527126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:43.527741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.528020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.528286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.528866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:43.530263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:43.539262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:43.540432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:43.541699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 637/1298] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:40:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:43.547172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:43.570254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:43.658513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3c1fc-c473-4f82-873e-b051c32b038d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:43.691781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:43.692383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.692661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.692921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.693583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:43.693886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:43.695243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:43.700565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 662/1299] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 19:40:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:43.707097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:43.740289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:43.809191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18ca69-265a-4def-a5eb-68c19318c0af None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:43.843588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:43.844203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.844592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.844871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.845606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] RBAC: Adding query filter params (interface=public, service_id=14cbb16fb5dd4b4cb6652fd5731b1e8c, region_id=RegionOne) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:43.845905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:43.847272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36964d1d-0215-4f4e-b2d4-56b549b70170 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:43.852414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 638/1300] 10.222.0.18 () {56 vars in 1251 bytes} [Tue Sep 13 19:40:43 2022] GET /identity/v3/endpoints?service_id=14cbb16fb5dd4b4cb6652fd5731b1e8c&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:43.858523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:43.941838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:43.961173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9259c10-a118-4be3-a75a-8d0c55c40481 None admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:43.995584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:43.996189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:43.996470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:43.996740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:43.997380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:43.998743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798d0ec-657c-4888-a0c9-0c7528340e05 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:44.005680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 663/1301] 10.222.0.18 () {56 vars in 1087 bytes} [Tue Sep 13 19:40:43 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:40:46.103090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d118-f7a6-4aaf-95df-576523fd0637 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.103683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d118-f7a6-4aaf-95df-576523fd0637 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.103683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887d118-f7a6-4aaf-95df-576523fd0637 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.104952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 639/1302] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:46.110292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.110724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.110824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.129261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:46.130425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:46.197484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 664/1303] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:46.458752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d118-f7a6-4aaf-95df-576523fd0637 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.558880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:46.566226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887d118-f7a6-4aaf-95df-576523fd0637 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.601234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.601851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.602123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.602392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.602979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.604352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:46.613417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:46.614469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: ERROR keystone.server.flask.application Sep 13 19:40:46.615634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 640/1304] 10.222.0.18 () {56 vars in 1117 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:46.620803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.699953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:46.723811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdf4c24-83b6-4fd5-8439-603f216c59f8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.756737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.757374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.757652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.757921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] PATH_INFO: `/v3/services` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.758551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:46.758852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.760277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355633dd-45a3-4fd8-bcf9-c9927a890016 None admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.765308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 665/1305] 10.222.0.18 () {56 vars in 1116 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:46.771401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.874489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37654c1d-6e61-4d4e-9136-29538ba0b2f5 None admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.908487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48456388-170c-4233-ae87-c9085249d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.909113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456388-170c-4233-ae87-c9085249d277 None admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.909389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456388-170c-4233-ae87-c9085249d277 None admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.909661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48456388-170c-4233-ae87-c9085249d277 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.910316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48456388-170c-4233-ae87-c9085249d277 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.911677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48456388-170c-4233-ae87-c9085249d277 None admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.933578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 641/1306] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:40:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:40:56.762408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=30 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:56.928864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:51.649001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e169e-4905-4954-ae3b-175b0a0d262e None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:51.650060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e169e-4905-4954-ae3b-175b0a0d262e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:51.650578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e169e-4905-4954-ae3b-175b0a0d262e None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:51.660607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361293a-b391-4c3d-a1fb-362a19575867 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:51.661704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361293a-b391-4c3d-a1fb-362a19575867 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:51.662205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361293a-b391-4c3d-a1fb-362a19575867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:51.666060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 666/1307] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:41:51 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:51.782477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f361293a-b391-4c3d-a1fb-362a19575867 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:51.783693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f361293a-b391-4c3d-a1fb-362a19575867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:51.900988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 642/1308] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:52.193198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:52.193487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:52.193758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:52.195158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 667/1309] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 13 19:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:52.201835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c6bc8-562f-4851-aef3-48b396cbb794 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:52.202261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c6bc8-562f-4851-aef3-48b396cbb794 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:52.202641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452c6bc8-562f-4851-aef3-48b396cbb794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:52.284767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-452c6bc8-562f-4851-aef3-48b396cbb794 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:52.286196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-452c6bc8-562f-4851-aef3-48b396cbb794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:52.309628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:52.368739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 643/1310] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 13 19:41:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:52.378029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:52.379822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:52.503562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1905fb6-e07a-4f99-92be-d912237003aa None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:52.589279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:52.589948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:52.590244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:52.590519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:52.592133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:52.661711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:52.724008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:52.725494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9563a0c-938f-4aa6-8cae-3663754ee66d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:52.746198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 668/1311] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:55.236736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:55.237336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:55.237336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:55.238635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 644/1312] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:55.244233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97c7c-a49e-4cba-b11e-5054e569bf04 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:55.244667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97c7c-a49e-4cba-b11e-5054e569bf04 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:55.244760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf97c7c-a49e-4cba-b11e-5054e569bf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:55.331749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bdf97c7c-a49e-4cba-b11e-5054e569bf04 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:55.332987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bdf97c7c-a49e-4cba-b11e-5054e569bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:55.432995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 669/1313] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:55.713603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:55.716905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:55.828162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b978004-53a6-4daf-afde-8324b1fa9ce9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:55.904648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:55.907160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:55.907799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:55.908061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:55.908361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:55.910839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:56.027197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:56.028736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58912a04-65d4-4045-bc00-815c4cb78ae1 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:56.048702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 645/1314] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:41:58.505146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:58.505718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:58.505718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:58.507092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 670/1315] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:58.512518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fd35-675e-463a-adc7-b7b76d3ba19d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:58.512981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fd35-675e-463a-adc7-b7b76d3ba19d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:58.513074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1667fd35-675e-463a-adc7-b7b76d3ba19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:58.580263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1667fd35-675e-463a-adc7-b7b76d3ba19d None None] MFA Rules not processed for user `51e799da9248476094c6ad78ce4252ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:58.581502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1667fd35-675e-463a-adc7-b7b76d3ba19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:58.635441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:58.681487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 646/1316] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:58 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:41:58.960339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:58.961985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:58.972291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:59.072257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b3c35-dcd4-408a-ad96-128276792003 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:59.148917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:59.149548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:59.149827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:59.150094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:59.151702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:59.214480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:59.262319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:59.263758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb4b69e-6265-468e-854f-3986b5388575 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:59.281900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 671/1317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:41:58 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:01.976369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4000233-3994-4865-af46-16a00a87cc6a None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:01.977009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4000233-3994-4865-af46-16a00a87cc6a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:01.977009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4000233-3994-4865-af46-16a00a87cc6a None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:01.978273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 647/1318] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:01.985679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76855d8-e401-4a0e-9e8a-6707b6b01264 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:01.986188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76855d8-e401-4a0e-9e8a-6707b6b01264 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:01.986554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76855d8-e401-4a0e-9e8a-6707b6b01264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.064948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b76855d8-e401-4a0e-9e8a-6707b6b01264 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:02.066216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b76855d8-e401-4a0e-9e8a-6707b6b01264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:02.173946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 672/1319] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:02.472722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:02.473011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:02.473279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.475318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 648/1320] 10.222.0.18 () {54 vars in 965 bytes} [Tue Sep 13 19:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:02.481425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189eb1d-92d6-422c-a9be-4abc3aa1fd90 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:02.481707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189eb1d-92d6-422c-a9be-4abc3aa1fd90 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:02.481968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189eb1d-92d6-422c-a9be-4abc3aa1fd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.505928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:02.571110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7189eb1d-92d6-422c-a9be-4abc3aa1fd90 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:02.572839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7189eb1d-92d6-422c-a9be-4abc3aa1fd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:02.654032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 673/1321] 10.222.0.18 () {58 vars in 1062 bytes} [Tue Sep 13 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:02.664095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:02.665680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:02.785621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe75615-41de-4bfd-91b6-1237de12c27b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:02.863431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:02.864061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:02.864331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:02.864599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:02.866247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:02.873569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:02.988585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:02.990057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef653e83-706e-436e-a62e-40d0a1dbfb88 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:03.009742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 649/1322] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:06.346498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:06.347122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:06.347122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:06.348604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 674/1323] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:06.354069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87313f7c-9a37-409e-ae0f-aa67eba82547 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:06.354550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87313f7c-9a37-409e-ae0f-aa67eba82547 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:06.354650 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87313f7c-9a37-409e-ae0f-aa67eba82547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:06.439560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-87313f7c-9a37-409e-ae0f-aa67eba82547 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:06.440826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-87313f7c-9a37-409e-ae0f-aa67eba82547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:06.506973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:06.559866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 650/1324] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:06.856913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:06.858480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:06.969496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46d0c7-7edd-4cd7-9ff6-a47119d40467 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:06.982481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:07.053726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:07.054347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:07.054617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:07.054877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:07.056444 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:07.187648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:07.189470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a41b1f-af51-45b8-8d1f-6a5adec2770b service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:07.209926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 675/1325] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:09.418524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:09.419164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:09.419164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:09.420407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 651/1326] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:09.425994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289b9d-3b87-4d49-a8c6-0c5c511fe953 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:09.426509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289b9d-3b87-4d49-a8c6-0c5c511fe953 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:09.426509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69289b9d-3b87-4d49-a8c6-0c5c511fe953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:09.477549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:09.499687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-69289b9d-3b87-4d49-a8c6-0c5c511fe953 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:09.501126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-69289b9d-3b87-4d49-a8c6-0c5c511fe953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:09.611807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 676/1327] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:09.888699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:09.890271 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:09.997713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bf3255-a568-4c89-b12e-8f87cf1bb15b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:10.079066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:10.079668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:10.079931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:10.080209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:10.081856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:10.099933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:10.201287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:10.202743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290c9b2-55b9-4ef7-992b-8a9d7ce74f3b service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:10.222839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 652/1328] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:12.451094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:12.451613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:12.451613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:12.452939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 677/1329] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:12.458425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23bac5-ccac-4067-952a-1a88a3961f82 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:12.458845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23bac5-ccac-4067-952a-1a88a3961f82 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:12.458971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23bac5-ccac-4067-952a-1a88a3961f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:12.529224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fd23bac5-ccac-4067-952a-1a88a3961f82 None None] MFA Rules not processed for user `51e799da9248476094c6ad78ce4252ce`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:12.530390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fd23bac5-ccac-4067-952a-1a88a3961f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:12.600435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:12.632676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 653/1330] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:12.915242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:12.916795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:13.036310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30942cbb-b7aa-4838-b76d-eea908fa676c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:13.062919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:13.127915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:13.128538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:13.128817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:13.129118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:13.131047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:13.263037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:13.264505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334e52f5-45c1-42c8-b1d6-c67dabf1ae85 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:13.285245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 678/1331] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:15.723191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:15.723973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:15.723973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:15.725145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 654/1332] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:15.731296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526642ed-611a-4e0e-a0d8-ad4c1d762f38 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:15.731914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526642ed-611a-4e0e-a0d8-ad4c1d762f38 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:15.731914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526642ed-611a-4e0e-a0d8-ad4c1d762f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:15.807529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:15.810983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-526642ed-611a-4e0e-a0d8-ad4c1d762f38 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:15.813240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-526642ed-611a-4e0e-a0d8-ad4c1d762f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:15.930571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 679/1333] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:16.214024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:16.215705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.339118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f99eb-ef66-4014-891b-874ae3903cc4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:16.428086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:16.428805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.429174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.429499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.431251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.466355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:16.561792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:16.563154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ec7a1e-1f8d-4384-a83a-de43a83e9f08 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:16.583859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 655/1334] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:18.833805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:18.834401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:18.834767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:18.837337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 680/1335] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:18.846629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bab48-d072-4107-ae2f-241e9395b26a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:18.847139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bab48-d072-4107-ae2f-241e9395b26a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:18.847581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bab48-d072-4107-ae2f-241e9395b26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:18.961381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7e0bab48-d072-4107-ae2f-241e9395b26a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:18.963480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7e0bab48-d072-4107-ae2f-241e9395b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:19.078386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:19.103417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 656/1336] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:19.625641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:19.627318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:19.757250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863d845-3fcf-4cd2-8d57-323c16d63299 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:19.788413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:19.835621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:19.836424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:19.836553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:19.836755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:19.838409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:19.956218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:19.957626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3aa660-2765-4a08-9250-1cedfae1fc00 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:19.978874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 681/1337] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:22.587332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:22.587936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:22.587936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:22.589439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 657/1338] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:22.595071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3d933-465d-4b11-a6a0-df2a0c172fa1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:22.595510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3d933-465d-4b11-a6a0-df2a0c172fa1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:22.595638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3d933-465d-4b11-a6a0-df2a0c172fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:22.662875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d6c3d933-465d-4b11-a6a0-df2a0c172fa1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:22.664081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d6c3d933-465d-4b11-a6a0-df2a0c172fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:22.673517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:22.769281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 682/1339] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:23.048246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:23.049847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.176188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadeba23-3978-48c7-94ba-7e112cba84fb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:23.260051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:23.260693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:23.261020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:23.261327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:23.262960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.304614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:23.380230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:23.381699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205f931-7c59-4786-9de8-91a1fcb14dd2 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:23.399837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 658/1340] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:27.762076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:27.762671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:27.762671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:27.764207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 683/1341] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:27.770500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3d8ff-36c6-42db-9b18-a019ce6cbd2c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:27.771096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3d8ff-36c6-42db-9b18-a019ce6cbd2c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:27.771190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3d8ff-36c6-42db-9b18-a019ce6cbd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:27.850582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-07a3d8ff-36c6-42db-9b18-a019ce6cbd2c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:27.851745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-07a3d8ff-36c6-42db-9b18-a019ce6cbd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:27.950153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:27.962324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 659/1342] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:28.243841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:28.245560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:28.382751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85889a-e006-4ac9-993a-910570e40f69 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:28.427760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:28.469391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:28.470012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:28.470283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:28.470549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:28.472148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:28.598442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:28.599887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67694718-4d46-4c6b-924b-35e9e01d423d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:28.620160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 684/1343] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:31.477714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:31.478323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:31.478409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:31.480387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 660/1344] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:31.486623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d5ac-1841-4b21-a54e-db3531af6cb3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:31.487311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d5ac-1841-4b21-a54e-db3531af6cb3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:31.487311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220d5ac-1841-4b21-a54e-db3531af6cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:31.568619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6220d5ac-1841-4b21-a54e-db3531af6cb3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:31.570437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6220d5ac-1841-4b21-a54e-db3531af6cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:31.592596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:31.684386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 685/1345] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:42:31.969122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:31.970717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:32.095785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa388ac1-0c83-48ae-9f32-682b6f0f9955 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:32.175265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:32.175881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:32.176157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:32.176431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:32.178096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:32.225723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:32.300846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:32.302550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248a5b1-8037-4ab8-a330-f26be53b3785 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:32.323340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 661/1346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:37.086016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:37.086623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:37.086623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:37.087951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 686/1347] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:42:37.094636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ea2a0-2f2c-4a07-97b1-7958bc7561c3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:37.095169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ea2a0-2f2c-4a07-97b1-7958bc7561c3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:37.095324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ea2a0-2f2c-4a07-97b1-7958bc7561c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:37.182514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9c0ea2a0-2f2c-4a07-97b1-7958bc7561c3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:37.184037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9c0ea2a0-2f2c-4a07-97b1-7958bc7561c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:37.301859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:37.306557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 662/1348] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:37.586958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:37.588741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:37.699517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a3114-9084-4154-b490-6a26274f35d2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:37.749733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:37.778074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:37.778691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:37.778958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:37.779244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:37.780862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:37.898285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:37.899771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e88e435-0c38-4178-9867-c2cb18bec9a4 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:37.923790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 687/1349] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:43.067662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736450-7c73-4794-a623-f23b2c0af81e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:43.067662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736450-7c73-4794-a623-f23b2c0af81e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:43.068148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e736450-7c73-4794-a623-f23b2c0af81e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:43.069731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 663/1350] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:43.083392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aa3d-66a1-427a-be2d-923f2e3b1274 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:43.083392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aa3d-66a1-427a-be2d-923f2e3b1274 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:43.083392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388aa3d-66a1-427a-be2d-923f2e3b1274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:43.226106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c388aa3d-66a1-427a-be2d-923f2e3b1274 None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:43.227408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c388aa3d-66a1-427a-be2d-923f2e3b1274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:43.255508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:43.381026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 688/1351] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:42:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:47.695482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a691-7a06-4d33-a142-1711f9d0492f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:47.696088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a691-7a06-4d33-a142-1711f9d0492f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:47.696478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31a691-7a06-4d33-a142-1711f9d0492f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:47.698027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 664/1352] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:42:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:47.703898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec06d-46a9-4de7-b9b6-41dc7165bc4d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:47.704373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec06d-46a9-4de7-b9b6-41dc7165bc4d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:47.704474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cec06d-46a9-4de7-b9b6-41dc7165bc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:47.783988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b1cec06d-46a9-4de7-b9b6-41dc7165bc4d None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:47.785125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b1cec06d-46a9-4de7-b9b6-41dc7165bc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:47.860446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:47.899269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 689/1353] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:42:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:49.393376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee896-327c-42fc-8045-88f235452dc4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:49.394007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee896-327c-42fc-8045-88f235452dc4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:49.394007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bee896-327c-42fc-8045-88f235452dc4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:49.395314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 665/1354] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:49.400661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc8bde-a4a0-4e61-937a-ead3840a1c8a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:49.401362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc8bde-a4a0-4e61-937a-ead3840a1c8a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:49.401362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc8bde-a4a0-4e61-937a-ead3840a1c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:49.487282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-debc8bde-a4a0-4e61-937a-ead3840a1c8a None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:49.488543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-debc8bde-a4a0-4e61-937a-ead3840a1c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:49.619886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 690/1355] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:42:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:53.017425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f937-591a-4100-bf7b-3efa5b904b82 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.018294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f937-591a-4100-bf7b-3efa5b904b82 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.018570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0f937-591a-4100-bf7b-3efa5b904b82 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.021022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 666/1356] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:53.026805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faac6a-b358-4db5-845e-91f85bd0ebfe None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.027305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faac6a-b358-4db5-845e-91f85bd0ebfe None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.027387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6faac6a-b358-4db5-845e-91f85bd0ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.088254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:53.140896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f6faac6a-b358-4db5-845e-91f85bd0ebfe None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:53.143032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f6faac6a-b358-4db5-845e-91f85bd0ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:53.288906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 691/1357] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:53.589788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.590347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.590794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.592312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 667/1358] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 19:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:53.601021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383364ce-2a6b-4f11-87d7-251c3549325b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:53.601301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383364ce-2a6b-4f11-87d7-251c3549325b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:53.601577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383364ce-2a6b-4f11-87d7-251c3549325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:53.701667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-383364ce-2a6b-4f11-87d7-251c3549325b None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:53.702840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-383364ce-2a6b-4f11-87d7-251c3549325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:53.772127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:53.860275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 692/1359] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 19:42:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:42:53.868537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:53.870199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.048112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3b5d9-2a5a-4b17-8535-ac99f9199ff9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:54.138066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:54.198871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:54.199391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.199674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.199958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.201694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.322410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:54.324286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ab98f-2f0d-45a3-8eac-798ed4dbddbb service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:54.345890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 668/1360] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:56.831439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:56.832129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:56.832129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:56.833545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 693/1361] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:56.838784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542cfb2-e903-4b1e-99f4-720965c42cf7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:56.839385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542cfb2-e903-4b1e-99f4-720965c42cf7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:56.839385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542cfb2-e903-4b1e-99f4-720965c42cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:56.915989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5542cfb2-e903-4b1e-99f4-720965c42cf7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:56.920003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5542cfb2-e903-4b1e-99f4-720965c42cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:56.966704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:57.038286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 669/1362] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:57.337623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:57.339724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:57.515824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc42a2-6543-4ab4-ad02-90be69d7e950 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:57.537470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:57.660017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:57.660639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:57.660976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:57.661252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:57.663212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:57.759083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:57.760747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00d4d3-386b-4540-bcb0-6708e9f8aa85 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:57.783924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 694/1363] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:00.273225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b63e0-105f-431d-b40e-dbe80b503096 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:00.273754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b63e0-105f-431d-b40e-dbe80b503096 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:00.274053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b63e0-105f-431d-b40e-dbe80b503096 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:00.275536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 670/1364] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:00.282547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef25fd-397c-4b6b-85b3-fbe581c6f6b4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:00.283024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef25fd-397c-4b6b-85b3-fbe581c6f6b4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:00.283125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef25fd-397c-4b6b-85b3-fbe581c6f6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:00.299336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:00.355433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-50ef25fd-397c-4b6b-85b3-fbe581c6f6b4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:00.356816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-50ef25fd-397c-4b6b-85b3-fbe581c6f6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:00.466836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 695/1365] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:00.760902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:00.761530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:00.761970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:00.763741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 671/1366] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 19:43:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:00.770755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb644083-3beb-4410-85aa-6372eae42c4a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:00.771122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb644083-3beb-4410-85aa-6372eae42c4a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:00.771588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb644083-3beb-4410-85aa-6372eae42c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:00.866687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cb644083-3beb-4410-85aa-6372eae42c4a None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:00.867928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cb644083-3beb-4410-85aa-6372eae42c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:00.905958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:00.997099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 696/1367] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 19:43:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:01.005066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:01.006634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:01.167420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b314b-7fd4-4fb2-a4cd-907b090806f1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:01.190296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:01.293424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:01.294842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:01.295223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:01.295594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:01.297522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:01.402210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:01.403984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9eedbf-87a8-4ec3-88b9-5c663148de02 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:01.418144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c892-e482-43a1-aae8-03350797e521 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:01.418706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c892-e482-43a1-aae8-03350797e521 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:01.418706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b2c892-e482-43a1-aae8-03350797e521 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:01.420008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 697/1368] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:01.425352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131efa-2cc9-48dc-b877-8a13a7177842 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:01.425630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131efa-2cc9-48dc-b877-8a13a7177842 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:01.425915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9131efa-2cc9-48dc-b877-8a13a7177842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:01.428932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 672/1369] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:01.502533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a9131efa-2cc9-48dc-b877-8a13a7177842 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:01.503829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a9131efa-2cc9-48dc-b877-8a13a7177842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:01.618161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:01.633138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 698/1370] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:01.957115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:01.957402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:01.957707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:01.959500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 673/1371] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 13 19:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:01.964339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ce4f-55ea-467f-8cf1-f7577c604b56 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:01.964623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ce4f-55ea-467f-8cf1-f7577c604b56 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:01.964869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307ce4f-55ea-467f-8cf1-f7577c604b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:02.055626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8307ce4f-55ea-467f-8cf1-f7577c604b56 None None] MFA Rules not processed for user `e306d2cfa2ce47b39520ccaf3daa6a10`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:02.056991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8307ce4f-55ea-467f-8cf1-f7577c604b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:02.137946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 699/1372] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 13 19:43:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:02.145544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:02.147132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:02.187161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:02.277338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8aa74-f02a-440f-955b-6be682a4c85d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:02.362613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:02.363327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:02.363603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:02.363777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:02.365376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:02.463171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:02.502980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:02.504427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b010434-302c-4dbb-98e5-5d62ae4d68f1 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:02.527217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 674/1373] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:03.678819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:03.679426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:03.679426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:03.680711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 700/1374] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:03.688469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dee75a-0354-4849-a0a1-9a382a21f9bb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:03.688995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dee75a-0354-4849-a0a1-9a382a21f9bb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:03.688995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dee75a-0354-4849-a0a1-9a382a21f9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:03.762488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b5dee75a-0354-4849-a0a1-9a382a21f9bb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:03.763609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b5dee75a-0354-4849-a0a1-9a382a21f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:03.866034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 675/1375] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:04.153397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:04.155044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:04.247484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:04.317388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120e82-fef5-4a17-9a86-fcfcf91d6960 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:04.452469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:04.452624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:04.452905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:04.453174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:04.454776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:04.508148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:04.553174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:04.554764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7aa82c-5fe1-4817-940b-fc7a07e18b6f service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:04.573458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 701/1376] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:43:06.990066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:06.990697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:06.990697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:06.992443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 676/1377] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:06.997834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d71125-5aed-4461-9c64-f46d14197ec4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:06.998293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d71125-5aed-4461-9c64-f46d14197ec4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:06.998374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d71125-5aed-4461-9c64-f46d14197ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:07.087250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a4d71125-5aed-4461-9c64-f46d14197ec4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:07.088625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a4d71125-5aed-4461-9c64-f46d14197ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:07.194726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 702/1378] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:07.478816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:07.480653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:07.529612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:07.664780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0aeaf-85b2-4733-891f-f4d95ecb0161 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:07.779042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:07.785381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:07.786529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:07.787209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:07.787579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:07.789418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:07.885954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:07.887618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03af21-b6c6-47f9-a22c-6611a6c85430 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:07.907120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 677/1379] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:10.174096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:10.174542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:10.174718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:10.181200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 703/1380] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:10.181598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525173-bc2d-4755-a605-5e92e9e6ec8e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:10.182097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525173-bc2d-4755-a605-5e92e9e6ec8e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:10.182097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d525173-bc2d-4755-a605-5e92e9e6ec8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:10.283276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5d525173-bc2d-4755-a605-5e92e9e6ec8e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:10.284501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5d525173-bc2d-4755-a605-5e92e9e6ec8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:10.357388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:10.390898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 678/1381] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:10.775140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:10.776726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:10.811880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:10.954600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee4826-b8fd-4688-8cab-fe6b1dd226cb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:11.068840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:11.086972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:11.087616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:11.087899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:11.088196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:11.089905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:11.202665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:11.204315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9321bc-c386-4d73-8ea0-3ebd2f219271 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:11.233442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 704/1382] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:13.510912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8281-caba-4281-9756-56894b95183e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:13.511572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8281-caba-4281-9756-56894b95183e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:13.511938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db8281-caba-4281-9756-56894b95183e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:13.513971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 679/1383] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:13.521241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2ef0-8c9d-4273-bf08-becc1b486f10 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:13.521716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2ef0-8c9d-4273-bf08-becc1b486f10 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:13.522240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2ef0-8c9d-4273-bf08-becc1b486f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:13.612205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-00cd2ef0-8c9d-4273-bf08-becc1b486f10 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:13.613541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-00cd2ef0-8c9d-4273-bf08-becc1b486f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:13.686383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:13.742313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 705/1384] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:14.030986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db8281-caba-4281-9756-56894b95183e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:14.032583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-15db8281-caba-4281-9756-56894b95183e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:14.190318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db8281-caba-4281-9756-56894b95183e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:14.248534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:14.318251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:14.318926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:14.319207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:14.319476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:14.321327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:14.428124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:14.429905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3bf443-f24d-4a49-81d8-331235c6281e service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:14.451597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 680/1385] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:16.668327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:16.668977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:16.668977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:16.670442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 706/1386] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:16.675733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0afb51-7e03-4fcc-91a1-2a0547a94a3b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:16.676206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0afb51-7e03-4fcc-91a1-2a0547a94a3b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:16.676292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0afb51-7e03-4fcc-91a1-2a0547a94a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:16.752503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ba0afb51-7e03-4fcc-91a1-2a0547a94a3b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:16.753720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ba0afb51-7e03-4fcc-91a1-2a0547a94a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:16.762445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:16.860597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 681/1387] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:17.144470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:17.146075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:17.301869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745153d-8e0b-4471-8b7f-af3a9de01e2c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:17.353622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:17.425950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:17.426575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:17.426855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:17.427124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:17.428739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:17.523707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:17.525356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431de8e-7344-49e1-a724-1840fdceabfd service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:17.544371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 707/1388] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:18.012513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99520a77-87f3-47e5-b8cb-6ff2deb822f0 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.013354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99520a77-87f3-47e5-b8cb-6ff2deb822f0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.013354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99520a77-87f3-47e5-b8cb-6ff2deb822f0 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.014469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 682/1389] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:18.020228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20140-7113-4554-897e-7c9a7be0bbdb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.020725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20140-7113-4554-897e-7c9a7be0bbdb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.020826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f20140-7113-4554-897e-7c9a7be0bbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.087955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.091351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-12f20140-7113-4554-897e-7c9a7be0bbdb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:18.093583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-12f20140-7113-4554-897e-7c9a7be0bbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.192627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 708/1390] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:18.493475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.493751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.493985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.495315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 683/1391] 10.222.0.18 () {54 vars in 966 bytes} [Tue Sep 13 19:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:18.501090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab4e4-701e-47d5-b26f-658de53c74bc None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.501520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab4e4-701e-47d5-b26f-658de53c74bc None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.501921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eab4e4-701e-47d5-b26f-658de53c74bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.585044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-04eab4e4-701e-47d5-b26f-658de53c74bc None None] MFA Rules not processed for user `2771f4c99b6d4cf3a23a5cdb7af09b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:18.586437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-04eab4e4-701e-47d5-b26f-658de53c74bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.659063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.665705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 709/1392] 10.222.0.18 () {58 vars in 1063 bytes} [Tue Sep 13 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:18.675246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:18.676852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.785484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b452f-ec8d-406a-a1b1-91a51c10ac70 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:18.825880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.865064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777769-5995-403b-83e0-e11272f2830e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:18.865683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777769-5995-403b-83e0-e11272f2830e service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.866203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777769-5995-403b-83e0-e11272f2830e service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.866467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777769-5995-403b-83e0-e11272f2830e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.868391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-83777769-5995-403b-83e0-e11272f2830e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.980757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777769-5995-403b-83e0-e11272f2830e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:18.982191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777769-5995-403b-83e0-e11272f2830e service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:19.000369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 684/1393] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:19.749530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299f49b-9042-4a3d-8a52-b2943a1e274b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:19.750061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299f49b-9042-4a3d-8a52-b2943a1e274b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:19.750061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299f49b-9042-4a3d-8a52-b2943a1e274b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:19.751426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 710/1394] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:19.754793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af354b9-7bbf-4bb4-b38c-2081e98a4816 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:19.755233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af354b9-7bbf-4bb4-b38c-2081e98a4816 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:19.755425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af354b9-7bbf-4bb4-b38c-2081e98a4816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:19.756610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6535e-e163-4934-bef4-591599f80386 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:19.756921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6535e-e163-4934-bef4-591599f80386 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:19.757196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6535e-e163-4934-bef4-591599f80386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:19.825927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5af354b9-7bbf-4bb4-b38c-2081e98a4816 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:19.827160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5af354b9-7bbf-4bb4-b38c-2081e98a4816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:19.827484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4fc6535e-e163-4934-bef4-591599f80386 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:19.828664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4fc6535e-e163-4934-bef4-591599f80386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:19.847032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:19.920571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 685/1395] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:19.930376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 711/1396] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:19.966482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:19.966743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:19.967008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:19.968362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 686/1397] 10.222.0.18 () {54 vars in 943 bytes} [Tue Sep 13 19:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:19.975472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:19.975754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:19.976020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.068874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] MFA Rules not processed for user `5ae58835c32e4a8f9e06f0a4c65ce26b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:20.070155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.089023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:20.184064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 712/1398] 10.222.0.18 () {58 vars in 1040 bytes} [Tue Sep 13 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:20.193656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:20.195202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.221397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:20.222604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.324378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656a8b11-5e0d-4711-9995-1b802b1fcdc6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:20.343463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c1fd3-f5ce-4cfc-8719-8597629de9c2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:20.370113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:20.372954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:20.451411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:20.452028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.452305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.452576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.454345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.457312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:20.457965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:20.458299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:20.458573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:20.460237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:20.554037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:20.555638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4313fcc9-29c5-4b4c-9bd7-2a9d776ea8f4 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:20.563473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:20.563947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 687/1399] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:43:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:43:20.565253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ec2d5-5b71-4042-ae5f-04668037dba8 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:20.588526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 713/1400] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:43:22.848237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.848871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.848871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.850727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 688/1401] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:22.856128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88a763-1136-4333-b4a0-0c90f60ebe25 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.856655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88a763-1136-4333-b4a0-0c90f60ebe25 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.856655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88a763-1136-4333-b4a0-0c90f60ebe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.911591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:22.926935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3c88a763-1136-4333-b4a0-0c90f60ebe25 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:22.928171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3c88a763-1136-4333-b4a0-0c90f60ebe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.041242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 714/1402] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:23.344547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.346832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.427026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.539212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d14e812-d655-4e1e-a9e2-c676724842e4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:23.702377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:23.703379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.703851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.704196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.706564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.734370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.825437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:23.827312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df6ebf-f6bd-4d07-b9e7-1bb5f45ff962 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:23.864502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 689/1403] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:26.806595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:26.807190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:26.807190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:26.808554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 715/1404] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:26.817380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c7b-c51c-45b8-a9a2-25fb675a9396 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:26.817380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c7b-c51c-45b8-a9a2-25fb675a9396 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:26.817380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c3c7b-c51c-45b8-a9a2-25fb675a9396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:26.901286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b91c3c7b-c51c-45b8-a9a2-25fb675a9396 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:26.902655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b91c3c7b-c51c-45b8-a9a2-25fb675a9396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.056438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:27.066331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 690/1405] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:27.356048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:27.358696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.573290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:27.617474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650924a3-2e90-484c-9d9a-6c6604168c4a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:27.791933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:27.792772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:27.793077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:27.793359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:27.795483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:27.941359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:27.952851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:27.955362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc928d8-c506-4011-8372-2916e02d1d5f service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:27.977264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 716/1406] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:30.454004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.454889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.454889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.455921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 691/1407] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:30.468247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ab68-6843-4dd3-b444-fa242587a2c7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:30.468247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ab68-6843-4dd3-b444-fa242587a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:30.468247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee9ab68-6843-4dd3-b444-fa242587a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:30.551221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cee9ab68-6843-4dd3-b444-fa242587a2c7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:30.553017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cee9ab68-6843-4dd3-b444-fa242587a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:30.692429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 717/1408] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:31.165859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:31.167453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:31.396489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca9d30d-bf5d-456c-b721-b0e424de5a3e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:31.480561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:31.532438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:31.533102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:31.533358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:31.533670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:31.535655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:31.639345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:31.641009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd74ece-b262-4159-9970-2b3864fb7966 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:31.660196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 692/1409] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:40.688717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=73 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.657755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=64 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:44.993578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0eec6-5da7-4adb-a9d2-5b55221c95a6 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:44.994662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0eec6-5da7-4adb-a9d2-5b55221c95a6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:44.994662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0eec6-5da7-4adb-a9d2-5b55221c95a6 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:44.996140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 718/1410] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 13 19:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:45.004992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2be4-6842-443f-b8db-18aeac71b788 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.005793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2be4-6842-443f-b8db-18aeac71b788 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.005793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef2be4-6842-443f-b8db-18aeac71b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.101956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3bef2be4-6842-443f-b8db-18aeac71b788 None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:45.103191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3bef2be4-6842-443f-b8db-18aeac71b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:45.216611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 693/1411] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 19:43:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:45.232302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0f69-e600-425b-9969-58c55b36f8f5 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.232555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0f69-e600-425b-9969-58c55b36f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.232818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c0f69-e600-425b-9969-58c55b36f8f5 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.234352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 719/1412] 10.222.0.18 () {54 vars in 918 bytes} [Tue Sep 13 19:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:45.239790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e06c1-e296-44bf-896d-66017110f02a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:45.240075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e06c1-e296-44bf-896d-66017110f02a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:45.240338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e06c1-e296-44bf-896d-66017110f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:45.323079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:45.327376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3e3e06c1-e296-44bf-896d-66017110f02a None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:45.329192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3e3e06c1-e296-44bf-896d-66017110f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:45.505012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 694/1413] 10.222.0.18 () {58 vars in 1015 bytes} [Tue Sep 13 19:43:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:48.212513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.213151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.213518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.216057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 720/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:43:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:48.225734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934ca5-1dfa-405a-b124-d49bae3b655c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:48.226423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934ca5-1dfa-405a-b124-d49bae3b655c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:48.226670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5934ca5-1dfa-405a-b124-d49bae3b655c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:48.315974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a5934ca5-1dfa-405a-b124-d49bae3b655c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:48.317362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a5934ca5-1dfa-405a-b124-d49bae3b655c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:48.390880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:48.440482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 695/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:48.733373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:48.735065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:48.922585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340132f-fed5-45c9-bd66-00c7f7766823 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:49.015320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:49.082989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:49.084000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:49.084466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:49.084936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:49.087915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:49.241043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:49.242696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d2374-ee91-4d40-9d69-29204d502373 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:49.265281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 721/1416] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:50.452449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.453141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.453141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.454798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 696/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:50.462617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6b7e-8875-4fb1-ab8f-4413f7def16c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.464137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6b7e-8875-4fb1-ab8f-4413f7def16c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.464600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb6b7e-8875-4fb1-ab8f-4413f7def16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.603726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-04fb6b7e-8875-4fb1-ab8f-4413f7def16c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:50.605848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-04fb6b7e-8875-4fb1-ab8f-4413f7def16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:50.654163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:50.733287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 722/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:50.885492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:50.887083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.062156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29103549-5e27-4bbe-ac24-4e88d65d28d2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:51.104958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:51.188264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:51.188920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:51.189198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:51.189469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:51.191231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.297277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:51.300116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e579f38-c370-4489-9a2c-6a357f95bfdd service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:51.333878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 697/1419] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:52.534749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.535511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.535511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.537028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 723/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:52.545818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac7f-2a34-4461-8c17-01f4f44d2606 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:52.546705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac7f-2a34-4461-8c17-01f4f44d2606 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:52.546705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7cac7f-2a34-4461-8c17-01f4f44d2606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:52.615779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:52.643514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1c7cac7f-2a34-4461-8c17-01f4f44d2606 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:52.644912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1c7cac7f-2a34-4461-8c17-01f4f44d2606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:52.787438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 698/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:52.949247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:52.952096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.149807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e7dcd5-f798-4777-b89e-8cf3ea8bb10b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:53.165932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:53.287947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:53.288596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:53.288874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:53.289192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:53.291002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:53.526464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:53.528271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363bcf8-c1f4-463d-909a-41578fe67c19 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:53.549650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 724/1422] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:54.798222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.798833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.799141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.801675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 699/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:43:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:54.811608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ad14-2081-425d-967f-a28d097eedfb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:54.812303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ad14-2081-425d-967f-a28d097eedfb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:54.812303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05ad14-2081-425d-967f-a28d097eedfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:54.838052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:54.906643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3b05ad14-2081-425d-967f-a28d097eedfb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:54.907906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3b05ad14-2081-425d-967f-a28d097eedfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:55.033148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 725/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:55.198652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:55.200427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:55.330542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.371845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe251b08-2c74-4e92-9412-f5b4b1f3112d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:55.532540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:55.533341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:55.533783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:55.534117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:55.535890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:55.625740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:55.642886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:55.644802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105780-b3f1-4d0f-9026-57fa5a0596f8 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:55.666656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 700/1425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:56.947910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.948477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.948477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:56.950026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 726/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:56.958556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac82d9-b89e-414a-aa98-9d0184c1d1c7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:56.959280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac82d9-b89e-414a-aa98-9d0184c1d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:56.959447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ac82d9-b89e-414a-aa98-9d0184c1d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.056300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-19ac82d9-b89e-414a-aa98-9d0184c1d1c7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:57.057680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-19ac82d9-b89e-414a-aa98-9d0184c1d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:57.181331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 701/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:57.333901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:57.336549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:57.448326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.524982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1da5f-4c1b-4988-ad1c-8bcdbe4c070b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:57.689174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:57.690222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:57.690705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:57.691181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:57.694295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:57.782264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:57.830352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:57.831956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f23aa7-2424-4e48-bef4-8f3c4b2570d4 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:57.851946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 727/1428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:58.951467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.952299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.952299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:58.953416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 702/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:58.961164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c71a-ea85-4ed3-86b6-8e45fd75252c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:58.961748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c71a-ea85-4ed3-86b6-8e45fd75252c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:58.961748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c71a-ea85-4ed3-86b6-8e45fd75252c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.057695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-20e3c71a-ea85-4ed3-86b6-8e45fd75252c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:59.058992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-20e3c71a-ea85-4ed3-86b6-8e45fd75252c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:59.174539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 728/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:59.325743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:59.327282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:59.381320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:59.492634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1de749-2172-4edc-8679-4c60bedb9e7e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:59.626945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:59.627635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:59.627946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:59.628254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:59.630145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:59.638538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:59.734209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:59.735868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94aeacc-0a9a-4647-9804-4be09c86cf26 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:59.756303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 703/1431] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:44:00.899041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:00.899926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:00.899926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:00.901676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 729/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:00.909855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c747a-0182-4516-a1d6-27bb51004733 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:00.910476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c747a-0182-4516-a1d6-27bb51004733 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:00.910476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3c747a-0182-4516-a1d6-27bb51004733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.039180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0b3c747a-0182-4516-a1d6-27bb51004733 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:01.041173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0b3c747a-0182-4516-a1d6-27bb51004733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.145808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.178165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 704/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:01.329695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:01.331392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.352435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.507156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfdab7c-50e3-4dd1-a59d-f8aeb1d20950 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:01.606539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.637005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:01.637624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:01.637898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:01.638139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:01.639983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:01.747886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:01.749566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d09754-6cc1-4eba-afe3-ffd76d3a8ab8 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:01.770006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 730/1434] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:02.878921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.879618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.879618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.880802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 705/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:02.888627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec22c9-2fd2-4a90-bca8-b7f6b86de9c0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:02.889294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec22c9-2fd2-4a90-bca8-b7f6b86de9c0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:02.889294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec22c9-2fd2-4a90-bca8-b7f6b86de9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:02.982672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-02ec22c9-2fd2-4a90-bca8-b7f6b86de9c0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:02.983957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-02ec22c9-2fd2-4a90-bca8-b7f6b86de9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:03.040927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:03.132406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 731/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:03.395647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:03.397384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:03.643499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da400df1-7af7-4252-aba4-94d3a8b66663 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:03.907068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:03.977172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:03.977797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:03.978065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:03.978326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:03.979956 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:04.079929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:04.081573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0b361-5e09-4b79-a866-a1eebc901cf4 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:04.101556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 706/1437] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:05.177544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.178428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.178428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.179431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 732/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:05.186659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051778-0f04-4dc9-b400-c96ad3771aff None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.187490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051778-0f04-4dc9-b400-c96ad3771aff None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.187490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a051778-0f04-4dc9-b400-c96ad3771aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.270982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4a051778-0f04-4dc9-b400-c96ad3771aff None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:05.272207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4a051778-0f04-4dc9-b400-c96ad3771aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:05.274530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.393643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 707/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:05.545928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:05.547548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:05.717401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6db4a0-751f-434a-aadd-b8d14d88eb57 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:05.754847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:05.878570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:05.879280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:05.879637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:05.879999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:05.882058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:05.994797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:05.996303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27384afe-80b8-4abd-87d4-123c95b0b094 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:06.018430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 733/1440] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:08.153762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.154677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.154677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.156648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 708/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:08.163683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f52a-5187-41fb-ad09-d8db7d7cc744 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.164315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f52a-5187-41fb-ad09-d8db7d7cc744 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.164315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681f52a-5187-41fb-ad09-d8db7d7cc744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.215143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.251588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1681f52a-5187-41fb-ad09-d8db7d7cc744 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:08.252839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1681f52a-5187-41fb-ad09-d8db7d7cc744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:08.376204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 734/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:08.667200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:08.670038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:08.800817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203c10e-8994-4a52-89e0-7caf744c7381 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:08.844320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.900982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.901631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.901905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.902166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.903912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.040191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:09.041723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8335e439-4968-46bf-8304-6075d05a6c5d service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:09.063518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 709/1443] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:11.762925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.763611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.763611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.765058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 735/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:11.773692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d445-bbcd-46c3-88b4-505e869e612b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.774494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d445-bbcd-46c3-88b4-505e869e612b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.774672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d445-bbcd-46c3-88b4-505e869e612b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.848171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5142d445-bbcd-46c3-88b4-505e869e612b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:11.849446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5142d445-bbcd-46c3-88b4-505e869e612b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:11.863924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:11.972218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 710/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:12.263061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:12.264710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:12.397716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0254473f-b1d2-4130-b3ea-86f759bcf958 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:12.412236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:12.484273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:12.484941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.485208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.485475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.487158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:12.615568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:12.617218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b60cde-f184-4234-bfd9-850d9c158fbf service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:12.642568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 736/1446] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:15.431817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.432527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.432527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.433732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 711/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:15.443672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ff37-e6cf-486f-9700-aa25bd4ab306 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.444250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ff37-e6cf-486f-9700-aa25bd4ab306 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.444552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa4ff37-e6cf-486f-9700-aa25bd4ab306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.530950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.553824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-baa4ff37-e6cf-486f-9700-aa25bd4ab306 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:15.555216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-baa4ff37-e6cf-486f-9700-aa25bd4ab306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:15.666010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 737/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:15.948092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.949711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:16.087484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced1733-273d-41a4-9c00-4c695cd0d027 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:16.137632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:16.181201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:16.181819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.182125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.182392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.184098 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:16.308391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:16.310054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aaed71-ce37-4a3d-9b78-a26ae9f1d4ca service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:16.329921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 712/1449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:18.740946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.741550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.741550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.743108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 738/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:18.749701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc04cf-6fd4-4608-918e-3394da4370fe None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.750566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc04cf-6fd4-4608-918e-3394da4370fe None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.750566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc04cf-6fd4-4608-918e-3394da4370fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.826129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d8dc04cf-6fd4-4608-918e-3394da4370fe None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:18.827369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d8dc04cf-6fd4-4608-918e-3394da4370fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:18.853435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:18.944160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 713/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:19.233198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:19.234858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:19.346611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29f586-45f4-4355-b575-d8fea0dab723 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:19.369482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:19.429637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:19.430400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:19.430736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:19.431092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:19.432798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:19.562000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:19.564769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb81d9-adc8-4d81-a9f5-ce61f540a8e9 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:19.598993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 739/1452] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:23.074586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.075578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.075578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.076556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 714/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:23.083245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029a7a-84b2-4d12-8089-9bba6335ec52 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.083827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029a7a-84b2-4d12-8089-9bba6335ec52 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.083827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f029a7a-84b2-4d12-8089-9bba6335ec52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.163244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.167212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-2f029a7a-84b2-4d12-8089-9bba6335ec52 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:23.169665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2f029a7a-84b2-4d12-8089-9bba6335ec52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:23.281242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 740/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:23.563478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:23.565077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:23.696537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83b1926-d2aa-4a27-8f38-6728ef8337e5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:23.761166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:23.791516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:23.792128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:23.792411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:23.792673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:23.794469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:23.928254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:23.929772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de88ee-9a95-43ea-9239-ad7c92d44e43 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:23.951964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 715/1455] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:44:26.263582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.264487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.264487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.266295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 741/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:26.272684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617fd3-a1a4-42c7-ac47-53dc42396346 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:26.273386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617fd3-a1a4-42c7-ac47-53dc42396346 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:26.273386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617fd3-a1a4-42c7-ac47-53dc42396346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:26.355919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-35617fd3-a1a4-42c7-ac47-53dc42396346 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:26.357386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-35617fd3-a1a4-42c7-ac47-53dc42396346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:26.394977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:26.481916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 716/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:26.761833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:26.763679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:26.912454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cafbb89-0ab0-49ee-b6cf-2036753187c1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:26.959432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.021563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:27.022222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.022642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.023505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.026172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:27.157021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:27.158490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d58f71a-5ffe-4934-93a0-dc28534151b7 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:27.180411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 742/1458] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:44:29.518787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.519677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.519677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.521246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 717/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:29.527303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b5b8-e09d-44c2-a6af-b8eb0cdac21c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:29.527902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b5b8-e09d-44c2-a6af-b8eb0cdac21c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:29.527902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b5b8-e09d-44c2-a6af-b8eb0cdac21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:29.601701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3425b5b8-e09d-44c2-a6af-b8eb0cdac21c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:29.603025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3425b5b8-e09d-44c2-a6af-b8eb0cdac21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:29.613070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:29.712991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 743/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:30.000983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:30.002630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:30.131460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05852d-6e9d-4ed2-9bd4-d2155e224b23 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:30.194233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.230195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:30.231224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.231705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.232184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.235297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:30.440089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:30.442892 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830417e7-50be-47c1-894b-0f0a341c6dbb service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:30.478434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 718/1461] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:44:32.793174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.793867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.793867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.795224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 744/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:32.801793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbff3-e9d4-472f-a613-eb59ccd287f4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:32.802384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbff3-e9d4-472f-a613-eb59ccd287f4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:32.802384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cbff3-e9d4-472f-a613-eb59ccd287f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:32.881808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e64cbff3-e9d4-472f-a613-eb59ccd287f4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:32.883066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e64cbff3-e9d4-472f-a613-eb59ccd287f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:32.933060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.007218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 719/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:33.292119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:33.293790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:33.447733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e50cd76-9d0d-4bd1-89aa-8107648b1847 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:33.504431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:33.550954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:33.551585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.551859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.552157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.554004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:33.698781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:33.700272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cf7390-0b91-4251-b6e7-6df990afd75f service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:33.722284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 745/1464] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:36.021052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:36.021718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:36.021718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:36.023326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 720/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:36.030322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea601-508e-48d2-8cff-e0de0ce7cf3c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:36.030804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea601-508e-48d2-8cff-e0de0ce7cf3c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:36.030911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ea601-508e-48d2-8cff-e0de0ce7cf3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:36.111538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-083ea601-508e-48d2-8cff-e0de0ce7cf3c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:36.112791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-083ea601-508e-48d2-8cff-e0de0ce7cf3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:36.133632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:36.234747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 746/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:36.524678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:36.526572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:36.671371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1267230-8e63-40a9-92ad-3a4bb20ea346 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:36.757806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:36.769075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:36.770087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:36.770562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:36.771023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:36.774117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:36.930142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:36.931638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983ac46-c8b1-4a00-a94e-d7001472a5ed service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:36.950979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 721/1467] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:39.196003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.196856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.196856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.198482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 747/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:39.205167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67103-efbe-4adc-908b-4f3686cee1da None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.205843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67103-efbe-4adc-908b-4f3686cee1da None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.205843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67103-efbe-4adc-908b-4f3686cee1da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.308932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dde67103-efbe-4adc-908b-4f3686cee1da None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:39.310225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dde67103-efbe-4adc-908b-4f3686cee1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:39.369824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.428441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 722/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:39.714349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:39.716122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:39.870030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6ccbbb-d337-485d-abc7-21898da3b4cc None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:39.928719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:39.974060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:39.974642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:39.975047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:39.975459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:39.977164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:40.144072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:40.145594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee897ff-b70d-408b-a394-e5075e2141f5 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:40.167502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 748/1470] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:49.426590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=22 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.164600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=66 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.211191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153348e8-f71e-41b2-bf73-1449ba285350 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.211835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153348e8-f71e-41b2-bf73-1449ba285350 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.212084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153348e8-f71e-41b2-bf73-1449ba285350 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.214588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 723/1471] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:01.220426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284b6a-ff6b-4c26-b26f-765ad1f1b917 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.221016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284b6a-ff6b-4c26-b26f-765ad1f1b917 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.221016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6284b6a-ff6b-4c26-b26f-765ad1f1b917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.295482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a6284b6a-ff6b-4c26-b26f-765ad1f1b917 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:01.296866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a6284b6a-ff6b-4c26-b26f-765ad1f1b917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:01.407126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 749/1472] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:01.706553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.706903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.707076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.708512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 724/1473] 10.222.0.18 () {54 vars in 966 bytes} [Tue Sep 13 19:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:01.714068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b38a-5cc5-454a-a777-7d586e8b26bc None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:01.714346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b38a-5cc5-454a-a777-7d586e8b26bc None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:01.714638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9353b38a-5cc5-454a-a777-7d586e8b26bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:01.799713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9353b38a-5cc5-454a-a777-7d586e8b26bc None None] MFA Rules not processed for user `2771f4c99b6d4cf3a23a5cdb7af09b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:01.800982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9353b38a-5cc5-454a-a777-7d586e8b26bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:01.823961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:01.894920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 750/1474] 10.222.0.18 () {58 vars in 1063 bytes} [Tue Sep 13 19:45:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:45:01.903700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:01.906895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:02.046852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98e15-12c0-4003-a5f4-6310fc2d6c0c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:02.125879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:02.126491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:02.126751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:02.127017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:02.128666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:02.189153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:02.256014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:02.257535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508df7c1-47b7-407c-90db-6ec603d5813a service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:02.278111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 725/1475] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:04.607983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.608625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.608625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.609971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 751/1476] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:45:04.616293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3ded-8614-4255-be37-6d8713a8e83f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:04.616953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3ded-8614-4255-be37-6d8713a8e83f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:04.616953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df3ded-8614-4255-be37-6d8713a8e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:04.698207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-83df3ded-8614-4255-be37-6d8713a8e83f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:04.699458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-83df3ded-8614-4255-be37-6d8713a8e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:04.809382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 726/1477] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:05.083191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:05.084740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:05.244399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a0153-37e9-4dbc-90cf-8465ca91ef7d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:05.307320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:05.355682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:05.356313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:05.356574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:05.356853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:05.357543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:05.359414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:05.359414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:45:05.359414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:45:05.360574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b60c8c-f6af-437b-9e48-f04f1082bc66 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:05.382299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 752/1478] 10.222.0.18 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:05 2022] GET /identity/v3/projects => generated 2553 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:45:07.674095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:07.674932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:07.674932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:07.676311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 727/1479] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:07.682634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8cdf6-05e2-4a55-b777-7b831a923824 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:07.683219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8cdf6-05e2-4a55-b777-7b831a923824 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:07.683219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a8cdf6-05e2-4a55-b777-7b831a923824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:07.763068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-87a8cdf6-05e2-4a55-b777-7b831a923824 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:07.764286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-87a8cdf6-05e2-4a55-b777-7b831a923824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:07.880445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 753/1480] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:45:08.181285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:08.183369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:08.194025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:08.392839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4554eb4-5ef7-4582-89d9-a9a4c438e283 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:08.479300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:08.513034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:08.513647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:08.513915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:08.514179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:08.515846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:08.616319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:08.617993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68143d5-d9c9-4cf6-a0b2-6af1ff20fb31 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:08.638724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 728/1481] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:45:11.095514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.096102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.096102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.097499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 754/1482] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:11.103445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1d12f-10b5-4076-a510-179d8843bff2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.103995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1d12f-10b5-4076-a510-179d8843bff2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.103995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1d12f-10b5-4076-a510-179d8843bff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.216769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-43a1d12f-10b5-4076-a510-179d8843bff2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:11.218400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-43a1d12f-10b5-4076-a510-179d8843bff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:11.274643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.331590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 729/1483] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:11.619085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:11.620756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:11.649959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.756146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd798b-40d5-448f-ae0d-0f01806c2ac4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:11.852713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:11.853388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:11.853665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:11.853932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:11.855727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:11.955433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:11.998236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:11.999932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c592b3e-427a-4a92-a3f1-6243e04d8914 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:12.021060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 755/1484] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:45:14.379680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.380280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.380280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.381684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 730/1485] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:14.389845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2435d-1126-409a-aa15-d5eca6ade748 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:14.390628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2435d-1126-409a-aa15-d5eca6ade748 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:14.390628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2435d-1126-409a-aa15-d5eca6ade748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:14.472135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c8e2435d-1126-409a-aa15-d5eca6ade748 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:14.473457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c8e2435d-1126-409a-aa15-d5eca6ade748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:14.585291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 756/1486] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:14.880298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:14.883274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:15.019004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9385a-3b13-49c8-ba6d-69df7e84a603 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:15.116181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:15.118161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:15.118881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:15.119181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:15.119860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:15.122354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:15.247415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:15.248913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73994bc5-7163-4784-8d65-ada86bb0e391 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:15.269069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 731/1487] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:17.830303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.831158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.831158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.832805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 757/1488] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:17.839179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79de83d-5fba-4ff4-94f6-84175f4ec87b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:17.839749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79de83d-5fba-4ff4-94f6-84175f4ec87b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:17.839953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79de83d-5fba-4ff4-94f6-84175f4ec87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:17.916630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a79de83d-5fba-4ff4-94f6-84175f4ec87b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:17.917912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a79de83d-5fba-4ff4-94f6-84175f4ec87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:17.976508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.026217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 732/1489] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:18.322697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:18.324690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:18.369363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.455957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbc7082-c365-46fd-ad83-ae992782ff1e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:18.545839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:18.546645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:18.546928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:18.547200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:18.548940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:18.648218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:18.686484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:18.688015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917387bf-ffd2-46d9-a574-89004c657274 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:18.708249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 758/1490] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:21.200482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241053-80b8-49d9-879e-991309b916d7 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.201131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241053-80b8-49d9-879e-991309b916d7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.201131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72241053-80b8-49d9-879e-991309b916d7 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.202774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 733/1491] 10.222.0.18 () {54 vars in 927 bytes} [Tue Sep 13 19:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:21.209658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac56a1-bbe2-4beb-8d66-e89b4254e266 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.210279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac56a1-bbe2-4beb-8d66-e89b4254e266 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.210279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac56a1-bbe2-4beb-8d66-e89b4254e266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.292779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-beac56a1-bbe2-4beb-8d66-e89b4254e266 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:21.294317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-beac56a1-bbe2-4beb-8d66-e89b4254e266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:21.417714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 759/1492] 10.222.0.18 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:21.716050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72241053-80b8-49d9-879e-991309b916d7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:21.717626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-72241053-80b8-49d9-879e-991309b916d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:21.853598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72241053-80b8-49d9-879e-991309b916d7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:21.937819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:21.938477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:21.938929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:21.939286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:21.941236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:21.951190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.067579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.069060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed91de-d093-4183-8387-4cfc73b7ea6c service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:22.087805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 734/1493] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:31.415556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=82 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.085389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=50 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:37.583830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:37.584513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:37.584513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:37.665689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:37.666908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:37.767749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 760/1494] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:37.776141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:37.776701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:37.776701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:37.856383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:37.857553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:37.957177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 735/1495] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:37.962076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:37.963413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:37.974929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dded74-18a5-4220-a60b-e9e6a0b1c333 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.037919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:38.073861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.074471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.074744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.075005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.075643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:38.075940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.076969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.076969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:46:38.076969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:46:38.077850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.093831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 761/1496] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:46:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:38.102530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:38.112161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7230a-e55c-49f3-8dda-eb426e04c2e2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.173704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:38.209648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.210253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.210524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.210786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.211410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.213199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.213199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:46:38.213199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:46:38.214399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.276571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 736/1497] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:46:38 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:38.281014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:38.282574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:38.318901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d607b-10e4-4ae1-ae55-e7a09ab60b4b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.418228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.418745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.419022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.419288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.419916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.421346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.421346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:46:38.421346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:46:38.422554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.466260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 762/1498] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:46:38 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:38.470595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:38.472117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:38.535265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb3972b-d34d-40e0-8d24-9c4b8a61cd63 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.622075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:38.635469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.636078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.636349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.636628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.637286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.638301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.638301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:46:38.638301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:46:38.639231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.652086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 737/1499] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:46:38.656743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:38.658347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:38.670184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e906705-7266-44ef-8668-e0155cb8ca04 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.697015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:38.774277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.774890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.775162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.775426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.776064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.777103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.777103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:46:38.777103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:46:38.777979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c13598-acc3-48c9-aee5-2362690ca705 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.791326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 763/1500] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:46:38 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:46:38.796220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:38.797827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:38.835300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b57de9-0e1d-4b80-afd5-be7bf3e01524 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:38.941058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:38.941776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:38.942052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:38.942317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] PATH_INFO: `/v3/projects/55f733b447384719876265a78bcd32fc/users/38d9a34587014ffd89e4b42eb1bf83fa/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:38.966985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f733b447384719876265a78bcd32fc, user_id=38d9a34587014ffd89e4b42eb1bf83fa, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:38.968021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:38.968021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:46:38.968021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:46:38.969981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ade55c-5604-4813-a1b0-3c7d9e70678e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:38.995655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 738/1501] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:46:38 2022] PUT /identity/v3/projects/55f733b447384719876265a78bcd32fc/users/38d9a34587014ffd89e4b42eb1bf83fa/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:39.008127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ad32-a207-4055-b2f6-c8501a6b4648 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.008486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ad32-a207-4055-b2f6-c8501a6b4648 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.008842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39ad32-a207-4055-b2f6-c8501a6b4648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.068118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7a39ad32-a207-4055-b2f6-c8501a6b4648 None None] MFA Rules not processed for user `38d9a34587014ffd89e4b42eb1bf83fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:39.069387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7a39ad32-a207-4055-b2f6-c8501a6b4648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:39.149516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 764/1502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:46:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:39.163443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.163727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.163988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.165474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 739/1503] 10.222.0.18 () {54 vars in 964 bytes} [Tue Sep 13 19:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:39.170873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104588-4608-4894-813a-63b68b7dd8a8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.171160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104588-4608-4894-813a-63b68b7dd8a8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.171428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a104588-4608-4894-813a-63b68b7dd8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.207928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:39.257640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1a104588-4608-4894-813a-63b68b7dd8a8 None None] MFA Rules not processed for user `e306d2cfa2ce47b39520ccaf3daa6a10`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:39.258861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1a104588-4608-4894-813a-63b68b7dd8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:39.339389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 765/1504] 10.222.0.18 () {58 vars in 1061 bytes} [Tue Sep 13 19:46:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:46:39.348920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:39.350493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:39.405613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:39.462738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881bca9-0dc6-4822-aa4a-1189d7c6a3d4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:39.544564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:39.545214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.545854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.546120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.547727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:39.664874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:39.666414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:39.670658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:39.687394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 740/1505] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:39.736616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cabc-b389-46cd-bc98-0648fefa57b3 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.737138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cabc-b389-46cd-bc98-0648fefa57b3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.737243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cabc-b389-46cd-bc98-0648fefa57b3 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.738716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 766/1506] 10.222.0.18 () {52 vars in 830 bytes} [Tue Sep 13 19:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:39.742526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:39.743761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:39.848212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f52a37-bf33-429b-9d94-ee2d7b229496 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:39.945345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:39.945976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:39.946246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:39.946510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] PATH_INFO: `/v3/users/38d9a34587014ffd89e4b42eb1bf83fa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:39.959980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d9a34587014ffd89e4b42eb1bf83fa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:39.961045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:39.961045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:46:39.961045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:46:39.962262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:39.973962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:40.032039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d9a34587014ffd89e4b42eb1bf83fa'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:40.043584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d9a34587014ffd89e4b42eb1bf83fa'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:40.044097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d9a34587014ffd89e4b42eb1bf83fa'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:40.044522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d9a34587014ffd89e4b42eb1bf83fa'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:40.069534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d9a34587014ffd89e4b42eb1bf83fa'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:40.090296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 741/1507] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:46:39 2022] DELETE /identity/v3/users/38d9a34587014ffd89e4b42eb1bf83fa => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:40.095007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:40.095328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:40.095596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:40.154789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:40.155993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:40.235596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:40.248850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 767/1508] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:40.258828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:40.260100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:40.364935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f37707c-b4fd-4bb9-8195-1699d3f1190c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:40.422644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:40.442843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:40.443451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:40.443720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:40.443980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:40.445593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:40.564306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:40.565786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f552ef-6d5b-4d62-81fd-4335ef8444a7 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:40.584653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 742/1509] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:40.920205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:40.921817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:40.934003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc14b6-9eb9-44c3-9f4c-f92ed7433e43 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:41.033719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:41.034359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:41.034631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:41.034902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] PATH_INFO: `/v3/projects/55f733b447384719876265a78bcd32fc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:41.045252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f733b447384719876265a78bcd32fc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:41.046629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:41.046629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:46:41.046629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:46:41.047852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:41.107676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f733b447384719876265a78bcd32fc', 'request_id': 'req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:41.123571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f733b447384719876265a78bcd32fc', 'request_id': 'req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:41.133676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f733b447384719876265a78bcd32fc', 'request_id': 'req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:41.156695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] The token cache is being invalidate because project 55f733b447384719876265a78bcd32fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:46:41.157007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:41.157534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7ccd1a5a-0b5a-4a48-b8c0-e28228152645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:41.158854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 768/1510] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:46:40 2022] DELETE /identity/v3/projects/55f733b447384719876265a78bcd32fc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:50.582054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=62 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:51.154693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:10.583318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.583966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.583966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.585442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 743/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:10.595661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619bfc-e247-4b09-8d2a-b83f45802048 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:10.596239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619bfc-e247-4b09-8d2a-b83f45802048 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:10.596357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78619bfc-e247-4b09-8d2a-b83f45802048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:10.698769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-78619bfc-e247-4b09-8d2a-b83f45802048 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:10.700146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-78619bfc-e247-4b09-8d2a-b83f45802048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:10.847527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 769/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:11.007456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:11.009787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:11.142717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442470b6-2b7a-4a6c-a5e4-4621b549cc60 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:11.148152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:11.148770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:11.149082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:11.149361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:11.162172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:11.163832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119c61c-437d-4c1e-b536-ca071c627301 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:11.178085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 744/1513] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:12.225441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5672bb-99d1-4078-b76c-6d062295be65 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.226045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5672bb-99d1-4078-b76c-6d062295be65 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.226045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5672bb-99d1-4078-b76c-6d062295be65 None None] PATH_INFO: `/v3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.227343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 770/1514] 10.222.0.26 () {54 vars in 925 bytes} [Tue Sep 13 19:48:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:12.234295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9899d-e52a-4194-b140-e0b4a55ce44f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.234874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9899d-e52a-4194-b140-e0b4a55ce44f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.234874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9899d-e52a-4194-b140-e0b4a55ce44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.284214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24dee-8247-48c8-b054-2cd2d880ecb5 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.284450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24dee-8247-48c8-b054-2cd2d880ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.284637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c24dee-8247-48c8-b054-2cd2d880ecb5 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.286075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 771/1515] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:12.292726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.293110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.293383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.305859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bec9899d-e52a-4194-b140-e0b4a55ce44f None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:12.307088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bec9899d-e52a-4194-b140-e0b4a55ce44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:12.314743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:12.315916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:12.324199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 772/1516] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:12.374640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:12.379691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 745/1517] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 13 19:48:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:12.475065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:12.487970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab89ca-b883-4225-8807-037638cceab8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:12.494002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:12.494603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:12.494880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:12.495142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:12.503572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:12.505175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2994c-f353-4a64-9ec1-625ebe5cb734 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:12.514881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 773/1518] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:21.229037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cec8-6be8-4697-aeed-6475aecbc686 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:21.229604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cec8-6be8-4697-aeed-6475aecbc686 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:21.229604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cec8-6be8-4697-aeed-6475aecbc686 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:21.230918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 746/1519] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 13 19:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:21.236358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d21f7c-807e-4994-8abc-c012713fad0e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:21.236837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d21f7c-807e-4994-8abc-c012713fad0e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:21.236837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d21f7c-807e-4994-8abc-c012713fad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:21.266059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-34d21f7c-807e-4994-8abc-c012713fad0e None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:21.267245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-34d21f7c-807e-4994-8abc-c012713fad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:21.276791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 774/1520] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 13 19:48:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:21.285198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cec8-6be8-4697-aeed-6475aecbc686 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:21.300549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cec8-6be8-4697-aeed-6475aecbc686 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:21.306178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:21.306775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:21.307050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:21.307313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:21.316269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:21.317917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560e898-08f3-4711-be62-4a405699c122 service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:21.327684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 747/1521] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:24.030767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8aaca-954a-40e7-9e2d-dfabeb296d5f None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.031263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8aaca-954a-40e7-9e2d-dfabeb296d5f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.031388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8aaca-954a-40e7-9e2d-dfabeb296d5f None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.032705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:24.040439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb1ec-f177-4db0-9066-5687e9d20a9b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.041002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb1ec-f177-4db0-9066-5687e9d20a9b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.041002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eeb1ec-f177-4db0-9066-5687e9d20a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.062957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-81eeb1ec-f177-4db0-9066-5687e9d20a9b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:24.064213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-81eeb1ec-f177-4db0-9066-5687e9d20a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:24.074922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:24.234178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8aaca-954a-40e7-9e2d-dfabeb296d5f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:24.250758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8aaca-954a-40e7-9e2d-dfabeb296d5f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:24.256418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:24.257134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:24.257343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:24.257613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:24.266940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:24.268528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489c67a-0aa6-47da-8fa2-f5bb5b846de2 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:24.279688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 776/1524] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:48:25.365352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d750-bec5-4f4e-9bd4-b902e2da97a8 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:25.365972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d750-bec5-4f4e-9bd4-b902e2da97a8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:25.365972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588d750-bec5-4f4e-9bd4-b902e2da97a8 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:25.367236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:25.373711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf1ea-4a94-4bb5-9b82-35e9b4245ce4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:25.374185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf1ea-4a94-4bb5-9b82-35e9b4245ce4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:25.374305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bf1ea-4a94-4bb5-9b82-35e9b4245ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:25.394785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-109bf1ea-4a94-4bb5-9b82-35e9b4245ce4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:25.395974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-109bf1ea-4a94-4bb5-9b82-35e9b4245ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:25.404429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:25.553439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588d750-bec5-4f4e-9bd4-b902e2da97a8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:25.570192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588d750-bec5-4f4e-9bd4-b902e2da97a8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:25.576359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:25.576994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:25.577270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:25.577535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:25.586012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:25.587590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a4fc4c-19ac-49e3-a39e-199bf18b527c service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:25.596773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 750/1527] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:29.553855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.554512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.554512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.555775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:29.561397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb2f4-bffd-4dc6-952c-8077ebf4c5cf None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.561943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb2f4-bffd-4dc6-952c-8077ebf4c5cf None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.561943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eb2f4-bffd-4dc6-952c-8077ebf4c5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.583430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-620eb2f4-bffd-4dc6-952c-8077ebf4c5cf None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:29.584659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-620eb2f4-bffd-4dc6-952c-8077ebf4c5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:29.594038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:29.886190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:29.888091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:29.918339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:29.976438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135c253d-e1b4-471f-b579-21cab9a5835d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:29.980380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:29.981042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.981318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.981587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.991499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:29.992978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f210fb-3abf-4d25-8c00-378b61d46771 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:30.003001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 779/1530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:32.150857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.151450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.151450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.152653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:32.158309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c60d9-4f51-44b8-8561-4873358c6caa None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.158779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c60d9-4f51-44b8-8561-4873358c6caa None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.158886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c60d9-4f51-44b8-8561-4873358c6caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.180125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f27c60d9-4f51-44b8-8561-4873358c6caa None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:32.181351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f27c60d9-4f51-44b8-8561-4873358c6caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:32.190989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:32.477661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:32.479847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:32.494478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8c9cf-afa7-483b-bd5e-e68d2b2135d3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:32.498437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:32.499047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.499329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.499587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.509090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:32.510522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce79d4-be9f-4d56-8bb0-a461fc6c7b0a service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:32.520428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 753/1533] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:36.360074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:36.360710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:36.360710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:36.361969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:48:36.368004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec89b2-5c85-4d88-babb-79f38620d85f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:36.368539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec89b2-5c85-4d88-babb-79f38620d85f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:36.368539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec89b2-5c85-4d88-babb-79f38620d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:36.389093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-aeec89b2-5c85-4d88-babb-79f38620d85f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:36.390299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-aeec89b2-5c85-4d88-babb-79f38620d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:36.399079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:36.697606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:36.699642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:36.795582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c4e211-16f6-44b8-b806-a71842b84e44 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:36.799884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:36.800545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:36.800817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:36.801126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:36.811170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:36.812693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa4290-1628-4158-b6a3-33dc3ea23c73 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:36.825079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 782/1536] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:39.029441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:39.030156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:39.030156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:39.031334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:48:39.037500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aff81-7694-498c-9bf0-8d97371ae88c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:39.038013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aff81-7694-498c-9bf0-8d97371ae88c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:39.038194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580aff81-7694-498c-9bf0-8d97371ae88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:39.059703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-580aff81-7694-498c-9bf0-8d97371ae88c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:39.060942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-580aff81-7694-498c-9bf0-8d97371ae88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:39.069080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:39.380727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:39.382663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:39.401429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc1ade-ea7f-4341-ab6e-2b305f95f209 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:39.405881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:39.406679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:39.407017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:39.407375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:39.420432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:39.422275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e391024-4002-455c-9595-6f1015cf6c1a service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:39.435553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 756/1539] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:49.054229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=66 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:49.429536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=34 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:03.592315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.593058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.593058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.622208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:03.623456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.637998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 784/1540] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:03.647737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.648347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.648347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.686151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:03.687374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:03.697152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 757/1541] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:03.702380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.713983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ae0150-4946-4c42-b128-0b243b0bd3c8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.719990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.720620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.721431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.721706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.722368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:03.722782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.724530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.741093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 785/1542] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:03.749864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.760845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944f338-13ed-448b-9d33-5748abeff2f7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.769329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.769960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.770238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.770507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.771227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.773181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.838433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 758/1543] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.842868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:03.861380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ee4af-1eff-4f3c-a165-0aaee41c05f4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:03.949666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:03.950332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:03.950605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:03.950876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:03.951541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:03.953539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354db6-19ef-417c-b1e9-2660a3d9f747 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:03.985579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 786/1544] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:03 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:03.989661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.006723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfb76e-f292-41b1-88bf-6fadbda80d44 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.012100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.012928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.013197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.013468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.014129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.015776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.019194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ad18-3baa-47e1-a2c4-7bce22d88847 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.019478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ad18-3baa-47e1-a2c4-7bce22d88847 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.019750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7ad18-3baa-47e1-a2c4-7bce22d88847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.060591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 759/1545] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:04.069624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.076391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d0b7ad18-3baa-47e1-a2c4-7bce22d88847 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.077716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d0b7ad18-3baa-47e1-a2c4-7bce22d88847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.091267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5fc5b-16d0-49f7-bdfb-a27c67dd9f32 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.099376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 787/1546] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:04.108244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.108609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.108782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.113492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.114158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.114437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.114784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.115492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.118455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359fb82-45a1-4eb0-b2d9-bd9d67fa1af4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.144067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 760/1547] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:04.147828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.148240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.148511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.157295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.159158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.184400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 788/1548] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.185281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.197502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.199164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.212207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 761/1549] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.212864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.215303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7516f411-a1e5-454c-aaf7-dc0b1c1e94dd None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.227914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b4572-1895-4d20-8ca0-a9b51061a04f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.231186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.231869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.232142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.232409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] PATH_INFO: `/v3/projects/d86336ff2858481ea24231bfb10775f1/users/f98fd1481406494e995df1128897415f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.235296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.235930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.236207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.236480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.237209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:04.237454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.238479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:04.238479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:50:04.238479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:50:04.239362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2c8afa-ff21-402f-bedd-2e0eb546175b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.269369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 762/1550] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:04.270438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487dbff-8a33-4322-b12f-6e67bf7aab62 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.270724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487dbff-8a33-4322-b12f-6e67bf7aab62 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.271046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4487dbff-8a33-4322-b12f-6e67bf7aab62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.281874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d86336ff2858481ea24231bfb10775f1, user_id=f98fd1481406494e995df1128897415f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.282955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:04.282955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:50:04.282955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:50:04.284988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.306277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4487dbff-8a33-4322-b12f-6e67bf7aab62 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.307538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4487dbff-8a33-4322-b12f-6e67bf7aab62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.324393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 763/1551] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.325745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6846df-8998-4f2e-8699-9905d37ef0d5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.326027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6846df-8998-4f2e-8699-9905d37ef0d5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.326368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6846df-8998-4f2e-8699-9905d37ef0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.331315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 789/1552] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:04 2022] PUT /identity/v3/projects/d86336ff2858481ea24231bfb10775f1/users/f98fd1481406494e995df1128897415f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:04.332245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.362132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5f6846df-8998-4f2e-8699-9905d37ef0d5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.363372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5f6846df-8998-4f2e-8699-9905d37ef0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.433286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e2f5a-67c1-41b6-a8a9-a66f7baa4232 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.437340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 764/1553] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.442230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.442428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c30ab-6be8-4ec6-afab-9c0fd3653ed5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.442657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c30ab-6be8-4ec6-afab-9c0fd3653ed5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.443019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430c30ab-6be8-4ec6-afab-9c0fd3653ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.443687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.443961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:50:04.448930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f212e4a-0eb5-403c-a216-df88e4ab2686 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.466805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-430c30ab-6be8-4ec6-afab-9c0fd3653ed5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.467876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-430c30ab-6be8-4ec6-afab-9c0fd3653ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.476340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 765/1554] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.477546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41907e-9da3-4bd6-b2ef-b352b1dd2b66 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.477833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41907e-9da3-4bd6-b2ef-b352b1dd2b66 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.478103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41907e-9da3-4bd6-b2ef-b352b1dd2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.483737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 790/1555] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/projects => generated 389 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:04.485316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68f95-b6a7-4bc8-9a92-0ab562af544c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.485601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68f95-b6a7-4bc8-9a92-0ab562af544c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.485873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68f95-b6a7-4bc8-9a92-0ab562af544c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.498692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4e41907e-9da3-4bd6-b2ef-b352b1dd2b66 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.499912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4e41907e-9da3-4bd6-b2ef-b352b1dd2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.507641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c9a68f95-b6a7-4bc8-9a92-0ab562af544c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.508944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c9a68f95-b6a7-4bc8-9a92-0ab562af544c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.521776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:04.561562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 766/1556] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.562919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.563205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.563479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.576696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 791/1557] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.578038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c01ab-28df-4913-a5e2-184846930cea None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.578321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c01ab-28df-4913-a5e2-184846930cea None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.578587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c01ab-28df-4913-a5e2-184846930cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.590495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.591721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.599276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-196c01ab-28df-4913-a5e2-184846930cea None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.600008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 767/1558] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.600429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-196c01ab-28df-4913-a5e2-184846930cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.600906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.605859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:04.612487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32c54e-4bae-48f9-bd30-d450ab33aded None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.613662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 792/1559] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.614843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.615119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.615405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.624000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.624722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.625184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.625462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.626121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:04.626423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.628087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.639103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.640317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.640688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 768/1560] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:04.641578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.652824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 793/1561] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.653746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.657595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65416406-fa23-40fe-90d3-e550aaf53aa1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.663036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.663905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.664154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.664425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.665103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.666126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:04.666126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:50:04.666126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:50:04.667336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.693518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 769/1562] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/users => generated 295 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:04.693518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.703470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b48ee-c32f-4733-ba8b-5e8db0aaa42d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.708050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.708744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.709057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.709328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.710013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:04.710317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.711939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82a4e3-08e6-49db-852b-bf653590280f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.712368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c8c552-bade-484e-9567-800e82d41c54 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.716523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.717163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.717440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.717704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.729294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 770/1563] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:04.729294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458196a0-e23a-441b-9ac2-d221888a1ecd None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.729294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458196a0-e23a-441b-9ac2-d221888a1ecd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.729294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458196a0-e23a-441b-9ac2-d221888a1ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.734194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.735640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c98a-00cc-43fc-ae5b-784a91d9ea79 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.745849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-458196a0-e23a-441b-9ac2-d221888a1ecd None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.746024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 794/1564] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:04.747165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-458196a0-e23a-441b-9ac2-d221888a1ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.747326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.747493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.747754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.755638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 771/1565] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.756854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bf82-4aed-4bc3-b4d2-8153b94ed615 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.757176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bf82-4aed-4bc3-b4d2-8153b94ed615 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.757444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3bf82-4aed-4bc3-b4d2-8153b94ed615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.771687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.773051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.780275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-80b3bf82-4aed-4bc3-b4d2-8153b94ed615 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.781786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-80b3bf82-4aed-4bc3-b4d2-8153b94ed615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.782144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 795/1566] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.785927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.791453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 772/1567] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.794101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.794459 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.794797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.798150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207b72f-4987-429a-b99f-0bb1cf7c00f8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.803583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.804420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.804862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.805183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.805827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.807731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.816108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:04.817398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:04.829235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 773/1568] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:04.833429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.842229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 796/1569] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:04.842854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5a78f-0dd9-40dc-a4bc-061b46c2f52e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.843162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.910838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.911465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.911871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.912140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.913095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.915414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.918644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d2b77-df63-4437-9cde-37f55aa6690d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.924326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.925060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.925546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.925858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.926586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.930187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 774/1570] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:04.930187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.930554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.942107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7496496-c70a-47ac-8c23-24322e1f2ef8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.950153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:04.950153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:04.950153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:04.951551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:04.951551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:04.951551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:04.953309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:04.969020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 775/1571] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:04.970243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.977732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 797/1572] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] POST /identity/v3/projects => generated 377 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:04.978592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:04.985083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40571411-c42a-4324-a97d-924019f4e06a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:04.985456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:05.055076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.055895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.056321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.056472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.057156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:05.057551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.059338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.067984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361c9e-c221-4b3e-b758-20e35ff5451b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.076944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 776/1573] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:05.076944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.078569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.079206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.079473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.079733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.080381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:05.080683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.082476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.093222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0a80-10d2-4018-9271-862fbf3c31ce admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.098334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 798/1574] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:05.098638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.099397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.099522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.099670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.099940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.100583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:05.100907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.103205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.114354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fadd3-21e1-480d-8cb2-ae472a3b12e6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.123573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 777/1575] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:05.123573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.125429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.126140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.126481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.127429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.131200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:05.131380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.134629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.138331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe3215-3fb4-4f57-9efa-6b49f8dd7841 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.144695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.146437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.146728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.147119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.147661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.149332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.163932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 778/1576] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.165324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.170535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 799/1577] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 302 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.171711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.181369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450bffc-1b87-44ed-815a-4f7dcae70e34 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.184466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76929568-39cd-41a0-919d-d5178aeb7ecb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.187118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.187932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.188214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.188544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.189267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.190614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.191273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.191629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.192062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.192518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.193506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.195679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.226383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 800/1578] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.227723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.233119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 779/1579] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.234174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.239403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc98d09-3893-48a0-8741-f591e9099475 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.306834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.307842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.308139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.308536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.308906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdbc879-3328-4687-8a7b-8057ad8620a0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.309225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.311364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.314488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.315253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.315667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.315992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.316686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.318960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.343178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 801/1580] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/projects => generated 371 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.344098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.350543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 780/1581] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/projects => generated 357 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.351484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.401318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:05.419602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3038d8-5f4a-4e30-b9e2-30d4ef7063fa admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.424474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.425230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.425387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.425656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.426291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.426766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada1f58-c756-4ae9-8b3e-9ca6cef6448c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.428369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.431841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.432505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.435399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.435399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] PATH_INFO: `/v3/projects/cd32f4615dd0450f8cc175e0bfcfc4a1/users/dbea9a6a35c6445a9ab76ee7278abda4/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.452141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd32f4615dd0450f8cc175e0bfcfc4a1, user_id=dbea9a6a35c6445a9ab76ee7278abda4, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.455482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.470662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 802/1582] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.470662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.478269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 781/1583] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:05 2022] PUT /identity/v3/projects/cd32f4615dd0450f8cc175e0bfcfc4a1/users/dbea9a6a35c6445a9ab76ee7278abda4/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:05.479097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.550538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660c788-0d7a-4c67-bed1-9839366ec28a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.556272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.557134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.557316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.557585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.557916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:05.558286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.560000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.573899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 803/1584] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.577722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.591389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99611155-3a92-4498-b684-7a7161978a73 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.591729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95867dc-912f-4767-8ca5-9dcfbbf5df0e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.599060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.599856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.600516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.600516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.601385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.602125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.602968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.603645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.603872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.604315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.605207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.607777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.622205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 782/1585] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.622205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.634189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915bcab2-b764-462d-bf27-e30bdabd797e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.640188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.641297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.641297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.641524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.645113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 804/1586] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 293 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.645288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.645376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.645467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.655432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46575bd0-9f94-428f-938a-b1f13c4ddb14 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.663978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.665029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.665374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.665781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.667766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.670022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d394bd9d-fa8d-4f56-9d45-62ae5bb346aa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.675273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 783/1587] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 286 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.676727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.691370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ce219e-14fc-4317-ba4e-0663f57cd6af admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.701807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 805/1588] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 279 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.703218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.703516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.703777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.706170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.708129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.708672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.709406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.710404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.714438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.728173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:05.729834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:05.744399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 806/1589] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:05.745709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.757732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ba238-602b-41d3-84f2-0eacbcc6b9c4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.761477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 784/1590] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:05 2022] POST /identity/v3/users => generated 296 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:05.761477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.763682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.764368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.764695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.765045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.766015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.768179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.771914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a940eb-0a43-4c8d-8814-550a07d246fa admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.778791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.779484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.779760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.780027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.780783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.783817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.784970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 807/1591] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.785888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.797066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db8050-396d-447f-b1e7-4b4aed7916e5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.799880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 785/1592] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.800774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.805688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.806464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.806737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.807129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.808364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.810036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.813795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534e00d-b518-43a0-b064-18f3a1bb7c6b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.819929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.820585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.820868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.821161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.821853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.822056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 808/1593] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.822891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.824993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.836721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5161e6-d467-42e6-a4d7-5b4580059e59 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.840982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 786/1594] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.840982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.846851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.847921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.848358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.848818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.849686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.851576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.870434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 809/1595] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.871392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.881543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4bf52-6da5-4c4a-863f-f87278affc96 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.886306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.886915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.887190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.887457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.888233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.889884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.907246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 810/1596] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:05.907246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.920754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074301-a034-469f-8876-8cf5c0fa69f5 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.921404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c870ec-3030-4c2e-a912-42cea5b7f0f0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:05.926052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.927024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:05.927397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.928220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:05.928220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.928438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] PATH_INFO: `/v3/projects/7774072241f54acf8a9b444a58075b02/users/0f550b80005846fd8e2f865962207456/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.929109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:05.930078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:05.949868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.950136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7774072241f54acf8a9b444a58075b02, user_id=0f550b80005846fd8e2f865962207456, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:05.951352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.952630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:05.963103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 787/1597] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:05.970420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 811/1598] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:05 2022] PUT /identity/v3/projects/7774072241f54acf8a9b444a58075b02/users/0f550b80005846fd8e2f865962207456/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:05.974508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.976546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:05.998605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2da6b9-74d9-4ea0-ba23-da835bd71d57 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.001613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:06.065699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166e8dfa-9ae7-44b6-9fd8-f28b74637bc5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.072705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.073315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.073594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.073861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.075178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.078608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.097898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 812/1599] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:05 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:06.099116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.110669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f689c337-d909-4b34-a5e9-e2c0710bcfb7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.116523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.117198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.121706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.122034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.122793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.125739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.140181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.141890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.142197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.142470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] PATH_INFO: `/v3/projects/3ee86defec834a3c9613ca3dfa5e4350/users/e60df169b5dc4aa59b5b0246544ef2b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.145882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 813/1600] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:06.146909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.161187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8efdb-3b0b-4dc9-a30f-84e490546962 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.166781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.167513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.167835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.168464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.169245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.171411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.182763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee86defec834a3c9613ca3dfa5e4350, user_id=e60df169b5dc4aa59b5b0246544ef2b6, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.185641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fea3b4-a90c-4aa3-9b4c-0b21dfe2a322 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.190199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 814/1601] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:06.190199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.205249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ac4b63-fa7c-44dd-9476-5764d66337b2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.216422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 788/1602] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:05 2022] PUT /identity/v3/projects/3ee86defec834a3c9613ca3dfa5e4350/users/e60df169b5dc4aa59b5b0246544ef2b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:06.216422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.216422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.216422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.224216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:06.257379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:06.260787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:06.306959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.308026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.308575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.308697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.311969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.314739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.332661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 815/1603] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:06 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:06.333922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.356777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e211c2-8fcf-4f58-bd40-4e711a23380f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.375466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.376409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.376821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.382540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] PATH_INFO: `/v3/projects/c23109b34b3c4f75ae636ba98d7b8396/users/abbec9edbc8741dfa4d47f60c64fc0f5/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.413935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 789/1604] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:06.414920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.425341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23109b34b3c4f75ae636ba98d7b8396, user_id=abbec9edbc8741dfa4d47f60c64fc0f5, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.428344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.435289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7bd3d-8414-4df7-88bd-3b81c16ebcf2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.443750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.445802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.446468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.446899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] PATH_INFO: `/v3/projects/8eb716b20b744990a62a8bd2e023d11c/users/0e901d4e55bd4b36b9c45ef4369613e7/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.453775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 816/1605] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:06 2022] PUT /identity/v3/projects/c23109b34b3c4f75ae636ba98d7b8396/users/abbec9edbc8741dfa4d47f60c64fc0f5/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:06.454679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.486229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb716b20b744990a62a8bd2e023d11c, user_id=0e901d4e55bd4b36b9c45ef4369613e7, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.490705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25372464-a5a5-4771-b0f7-0157d164f85e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.514372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 790/1606] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:06 2022] PUT /identity/v3/projects/8eb716b20b744990a62a8bd2e023d11c/users/0e901d4e55bd4b36b9c45ef4369613e7/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:06.515443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.515843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.516212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.556758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:06.558171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:06.618838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ac6fa-c347-497a-b33f-bf2f0ec85a99 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.643154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:06.701636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 791/1607] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:06.702623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.723029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc6616-3eee-445c-b821-ad41365936bc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.731406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.732153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.732430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.732706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] PATH_INFO: `/v3/projects/592bbccef3ef42ccbc7e2c3dc5bb86de/users/a1c9d4af0a98406692e1d79293844c68/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.832605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.832605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.832605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.832605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] PATH_INFO: `/v3/projects/261dad42b0bc43d6832b77f5850f18d9/users/f3bb91dc3f1347e6acd722b0af28ca32/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.833217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=592bbccef3ef42ccbc7e2c3dc5bb86de, user_id=a1c9d4af0a98406692e1d79293844c68, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.833217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.833217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 792/1608] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:06 2022] PUT /identity/v3/projects/592bbccef3ef42ccbc7e2c3dc5bb86de/users/a1c9d4af0a98406692e1d79293844c68/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:06.833217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:06.833671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] RBAC: Authorizing `identity:create_grant(project_id=261dad42b0bc43d6832b77f5850f18d9, user_id=f3bb91dc3f1347e6acd722b0af28ca32, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:06.833671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d2671-2e1b-4da7-8fc7-d9955bad524e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:06.848538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 817/1609] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:06 2022] PUT /identity/v3/projects/261dad42b0bc43d6832b77f5850f18d9/users/f3bb91dc3f1347e6acd722b0af28ca32/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:06.850361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b87c8-68de-4f45-83af-3075fdeb0565 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.851999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b87c8-68de-4f45-83af-3075fdeb0565 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.852377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b87c8-68de-4f45-83af-3075fdeb0565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.875265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05c2a3-0053-4e79-a3f3-7f67d51786fb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:06.887684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:06.887939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-736b87c8-68de-4f45-83af-3075fdeb0565 None None] MFA Rules not processed for user `abbec9edbc8741dfa4d47f60c64fc0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:06.890013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-736b87c8-68de-4f45-83af-3075fdeb0565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:06.981477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:06.982185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.982626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.982902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:06.994619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 818/1610] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:06.996271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf42bf1-9a56-4782-a76c-bdc10c7a6cf5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:06.996580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf42bf1-9a56-4782-a76c-bdc10c7a6cf5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:06.997024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf42bf1-9a56-4782-a76c-bdc10c7a6cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.054922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3bf42bf1-9a56-4782-a76c-bdc10c7a6cf5 None None] MFA Rules not processed for user `0e901d4e55bd4b36b9c45ef4369613e7`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:07.058026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3bf42bf1-9a56-4782-a76c-bdc10c7a6cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:07.120133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.122654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.138978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 793/1611] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:07.140007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.175452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dcb1a-57c2-43ce-90a0-3814f68e52be service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.182542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:07.183297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.183631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.183902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.202462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.204120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5019c4f-33f3-4374-8864-9221e583e576 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.226659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 794/1612] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:07.228054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b503e3-a979-4ac4-a2a3-882402ed0b9b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.228405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b503e3-a979-4ac4-a2a3-882402ed0b9b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.228706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b503e3-a979-4ac4-a2a3-882402ed0b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.229376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 819/1613] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:07.230985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.231274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.231598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.257904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-47b503e3-a979-4ac4-a2a3-882402ed0b9b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:07.259946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-47b503e3-a979-4ac4-a2a3-882402ed0b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:07.268006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] MFA Rules not processed for user `f3bb91dc3f1347e6acd722b0af28ca32`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:07.269335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:07.274601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:07.276147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 795/1614] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:07.277984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d24e0-1bba-4069-8861-df764f77b7d4 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.278264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d24e0-1bba-4069-8861-df764f77b7d4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.278519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d24e0-1bba-4069-8861-df764f77b7d4 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.281177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 796/1615] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 19:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:07.282994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d24e0-1bba-4069-8861-df764f77b7d4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.300929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d24e0-1bba-4069-8861-df764f77b7d4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.308515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:07.309918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.310281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.310704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.311429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.314728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87112a-0d31-4cfe-8bd1-ea974db8cc7a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.366014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 797/1616] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:07.366014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.366014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.366014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.405532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 820/1617] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:07.406594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.429376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ef1e1a-3965-4e89-95db-da35b7d1f5ce None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.454547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:07.457389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:07.615794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:07.621055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.621679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.629002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.656424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:07.689444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 798/1618] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:07.691478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.804866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.806816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.831469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 821/1619] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:07.832317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.853117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42874561-cc20-45b0-8e4b-33612a741682 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.869743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:07.870129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:07.870487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:07.870853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:07.871605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:07.873867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:07.909241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 822/1620] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/projects => generated 355 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:07.910873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:07.925758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c91268-b179-430d-b342-7c26d4c42675 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:07.962631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:08.041721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda22c79-d1a7-4138-a979-b6e24f374b8c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.049052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.049793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.050202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.051543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.083106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.083466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.083677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.083949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.085196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.087219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.137562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 799/1621] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:07 2022] POST /identity/v3/users => generated 293 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:08.138585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.151750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1948b-6f07-430e-a8a9-8535d48920fb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.158363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.159120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.159647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.159838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.160696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.163092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.181508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.185975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.205078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 800/1622] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:08 2022] POST /identity/v3/users => generated 284 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:08.206070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.220568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b77133-ee04-481e-ad3a-dd039991598c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.222208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 823/1623] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:08.223300 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.227479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.228812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.229286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.229702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.230512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.232291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.247506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53341941-50ae-45a2-9fb9-d66f21e92e2d service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.253029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.253828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.254207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.254576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.255794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 801/1624] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:08.256330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.258360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.261560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.271433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 824/1625] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:08.275422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.288919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72601a4-8e00-4852-8139-1e6cd32f6fac admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.295037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.296183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.296660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.297373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] PATH_INFO: `/v3/projects/44627e8020284298ba49f1e82b589e79/users/2362d34ebd73470f9515503e081dcdfa/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.304217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f917bd5-d6cf-4bea-b397-10ed1c74c61c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.320149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] RBAC: Authorizing `identity:create_grant(project_id=44627e8020284298ba49f1e82b589e79, user_id=2362d34ebd73470f9515503e081dcdfa, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.323432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.329376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.331502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.331785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.332110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] PATH_INFO: `/v3/projects/ef787d9f64da430b8575dc3b8c98954a/users/bd66195067b8426c8427a20c0a7aac63/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.345872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 825/1626] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:08 2022] PUT /identity/v3/projects/44627e8020284298ba49f1e82b589e79/users/2362d34ebd73470f9515503e081dcdfa/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:08.351228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.372497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef787d9f64da430b8575dc3b8c98954a, user_id=bd66195067b8426c8427a20c0a7aac63, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.378469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.407563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 802/1627] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:08 2022] PUT /identity/v3/projects/ef787d9f64da430b8575dc3b8c98954a/users/bd66195067b8426c8427a20c0a7aac63/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:08.413711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.454949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e952-e277-4509-9d58-6d8bc3f9274e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.518742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:08.530903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:08.557402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.559652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.560078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.560497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.560693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a4dd-1657-45cf-96c3-7e8603b8082c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.561479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.563776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.574110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.574748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.575021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.575287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.576101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.579334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.580932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 826/1628] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:08.585828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.608838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 803/1629] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:08 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:08.611081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78279da9-9470-4588-9d5a-401b9a70e6c2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.613206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:08.629179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.631849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2a1134-5c13-4d93-b919-5e774ecff7a4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:08.633765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.634048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.634322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] PATH_INFO: `/v3/projects/44627e8020284298ba49f1e82b589e79/users/2362d34ebd73470f9515503e081dcdfa/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.641994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=44627e8020284298ba49f1e82b589e79, user_id=2362d34ebd73470f9515503e081dcdfa, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.649475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8cd47-2ab5-4345-bd09-e639255eae0b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.658374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:08.659073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.659346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.659613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] PATH_INFO: `/v3/projects/ef787d9f64da430b8575dc3b8c98954a/users/bd66195067b8426c8427a20c0a7aac63/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.664368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef787d9f64da430b8575dc3b8c98954a, user_id=bd66195067b8426c8427a20c0a7aac63, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:08.672821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388fb2c-c402-4e17-90f7-294ebdfea592 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:08.703959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 827/1630] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:08 2022] PUT /identity/v3/projects/44627e8020284298ba49f1e82b589e79/users/2362d34ebd73470f9515503e081dcdfa/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:08.705979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58866b-d3d6-46e6-a99a-5c688c1e470f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.706264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58866b-d3d6-46e6-a99a-5c688c1e470f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.706530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58866b-d3d6-46e6-a99a-5c688c1e470f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.723296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 804/1631] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:08 2022] PUT /identity/v3/projects/ef787d9f64da430b8575dc3b8c98954a/users/bd66195067b8426c8427a20c0a7aac63/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:08.725637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.725921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.726187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.769871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1f58866b-d3d6-46e6-a99a-5c688c1e470f None None] MFA Rules not processed for user `2362d34ebd73470f9515503e081dcdfa`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:08.771236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1f58866b-d3d6-46e6-a99a-5c688c1e470f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:08.785573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] MFA Rules not processed for user `bd66195067b8426c8427a20c0a7aac63`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:08.786944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:08.959628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 828/1632] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:08 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:08.959628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec0fb-d562-480d-8026-5490f4a94613 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.959628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec0fb-d562-480d-8026-5490f4a94613 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.959628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec0fb-d562-480d-8026-5490f4a94613 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.960085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 829/1633] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 19:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:08.968429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 805/1634] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:08.969011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:08.969011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:08.969011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:08.979182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:09.016414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:09.018742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:09.100750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699eacf1-a419-4a70-973f-1d39b65f85b0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:09.113686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:09.114360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:09.114723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:09.115026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:09.133816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:09.135757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:09.142353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 830/1635] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:09.143476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:09.160719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee22d00-fee4-44d8-98e4-38eaa3b77aca None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:09.162631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 806/1636] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:09.164878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:09.165662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:09.166093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:09.166429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:09.179301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:09.181108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:09.195377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 831/1637] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:09.275640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:09.296599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef31b433-fca6-4286-a7d2-281af33b5ef5 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:09.312836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:09.314201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:09.315081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:09.317045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:09.356928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:09.360783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c275058a-e5a1-4fcc-bb4f-3fc602985cf7 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:09.387597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 807/1638] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:09.495292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:09.507129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2e0df-66a2-4c3b-9b5f-ecdfa1f9abe3 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:09.521330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:09.523860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:09.526822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:09.527304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:09.609457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:09.688787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:09.690293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:09.713644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 832/1639] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:10.076679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.077278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.077278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.139386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] MFA Rules not processed for user `0f550b80005846fd8e2f865962207456`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:10.141025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:10.219729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:10.240515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:10.252138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a30a1d6-6e2c-45e8-9486-448b69143d94 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:10.265497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:10.268362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.269004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.269558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.351543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 808/1640] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:10.352863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:10.445953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:10.449739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:10.510438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 833/1641] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:10.536136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10206d1-c445-4c43-aa5b-81a282585ff2 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:10.553646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:10.558667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:10.559146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:10.563145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:10.564535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:10.575489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:10.733551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:10.734743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 809/1642] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:10 2022] POST /identity/v3/projects => generated 405 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:10.782728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a193d-a16d-410f-96a7-8faf6a1f6411 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.070552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.071756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.072212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.072661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.073801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.081008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.173231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.196511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 834/1643] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:10 2022] POST /identity/v3/users => generated 309 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:11.237433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d1deed-8339-4aa4-9e6d-59b569a3c5b3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.249423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.249960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.252635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.255840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.256640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.258339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.291783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 810/1644] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:11.305515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.365417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b39e5c-0827-44d4-b65b-a58b40d89bf8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.389437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.390580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.391056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.393092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] PATH_INFO: `/v3/projects/9b65eaca28a840cea18272a521f5c91b/users/f4e468bf91564abb95f739fc00bd35d5/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.482806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b65eaca28a840cea18272a521f5c91b, user_id=f4e468bf91564abb95f739fc00bd35d5, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.493230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.562794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 835/1645] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:11 2022] PUT /identity/v3/projects/9b65eaca28a840cea18272a521f5c91b/users/f4e468bf91564abb95f739fc00bd35d5/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:11.563885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.577201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec20d03-3308-4fd0-961e-1bb744827bfe admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.841619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.842252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.842530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.842801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.851448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.853217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4707364-ca03-4fc5-a3b0-54e7508504f3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:11.897739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 811/1646] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:11.899497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:11.935939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b44eb-4362-4a7c-9176-1a942f4852b9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:11.948210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:11.958705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:11.958996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:11.959276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] PATH_INFO: `/v3/projects/9b65eaca28a840cea18272a521f5c91b/users/f4e468bf91564abb95f739fc00bd35d5/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:11.967791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b65eaca28a840cea18272a521f5c91b, user_id=f4e468bf91564abb95f739fc00bd35d5, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:11.971049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d545323-8373-44e5-8e4a-68fb65b65212 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:12.000274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 836/1647] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:11 2022] PUT /identity/v3/projects/9b65eaca28a840cea18272a521f5c91b/users/f4e468bf91564abb95f739fc00bd35d5/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:12.459483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399fd-7046-46dc-83b5-d2a8954e834f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.460075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399fd-7046-46dc-83b5-d2a8954e834f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.460075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf399fd-7046-46dc-83b5-d2a8954e834f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.461762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 812/1648] 10.222.0.18 () {54 vars in 962 bytes} [Tue Sep 13 19:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:12.471437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc83b5-bd7b-4233-92d0-d9e978d0f109 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.471726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc83b5-bd7b-4233-92d0-d9e978d0f109 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.471993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc83b5-bd7b-4233-92d0-d9e978d0f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.522081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f1dc83b5-bd7b-4233-92d0-d9e978d0f109 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:12.525333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f1dc83b5-bd7b-4233-92d0-d9e978d0f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:12.590571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:12.826594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 837/1649] 10.222.0.18 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:12.836390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3eacf-70f7-4074-b729-35dde66e066d None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.836673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3eacf-70f7-4074-b729-35dde66e066d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.837150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3eacf-70f7-4074-b729-35dde66e066d None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.839010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 813/1650] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 19:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:12.851288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda91ee-737f-423f-9524-e258bbdadf47 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:12.851906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda91ee-737f-423f-9524-e258bbdadf47 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:12.852385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eda91ee-737f-423f-9524-e258bbdadf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:12.888370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6eda91ee-737f-423f-9524-e258bbdadf47 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:12.889884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6eda91ee-737f-423f-9524-e258bbdadf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:12.913719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 838/1651] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:12.930675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3eacf-70f7-4074-b729-35dde66e066d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.072647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:13.104730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c3eacf-70f7-4074-b729-35dde66e066d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.110946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.128373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.128839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.129153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.156767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.158646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9016b930-a5d0-4b7c-ba9d-85124af10d2d service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.194045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 814/1652] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:14.137176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.137687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.137778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.265570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] MFA Rules not processed for user `f98fd1481406494e995df1128897415f`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.271940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.463753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 839/1653] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:14.475235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8263-1eb6-41d8-9ee2-305bc859fea7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.476839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8263-1eb6-41d8-9ee2-305bc859fea7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.477373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc8263-1eb6-41d8-9ee2-305bc859fea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.515132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-60cc8263-1eb6-41d8-9ee2-305bc859fea7 None None] MFA Rules not processed for user `f98fd1481406494e995df1128897415f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.518198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-60cc8263-1eb6-41d8-9ee2-305bc859fea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.541904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 815/1654] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:14.555080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.571847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f57a28-78ce-4ebc-b113-3bd6d6de82de None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.597309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.598242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.599949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.600479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.624978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.644211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040b254-dcb0-4ccf-80f9-7366f7ae889f service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.670247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 840/1655] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:15.230159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.232313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.233113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.269769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] MFA Rules not processed for user `f4e468bf91564abb95f739fc00bd35d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.271764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.349445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c6bd-2df2-4193-999f-cc3660ff1de7 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.350472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c6bd-2df2-4193-999f-cc3660ff1de7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.350953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f4c6bd-2df2-4193-999f-cc3660ff1de7 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.358068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 841/1656] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 19:50:15 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:15.365546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.365964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.366572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.374386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 816/1657] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.433003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.434741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.525714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 842/1658] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.558914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.576323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18c4a7-f25c-4b81-a8f6-d9900f7eb07c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.580677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.581623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.582103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.582562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.593891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.597828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ee295-c238-4f93-9df4-05378b64fd90 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.610487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 817/1659] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.845603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.898465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:17.099079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010fbc-4a7e-40b1-8151-88fbd65708b9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.117054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.117803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.118091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.118375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.127071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.192771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.194703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa95e46-0dec-4793-95e8-3d3374a40436 service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.260486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 843/1660] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:17.402461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3152-7d4a-44e5-83e2-40d85afa0fb9 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.403066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3152-7d4a-44e5-83e2-40d85afa0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.403427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f3152-7d4a-44e5-83e2-40d85afa0fb9 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.406930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 818/1661] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:50:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:17.427671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e98e2a-f69a-4a8d-ad89-89ac9c7e78f6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.428013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e98e2a-f69a-4a8d-ad89-89ac9c7e78f6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.428201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e98e2a-f69a-4a8d-ad89-89ac9c7e78f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.480510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b1e98e2a-f69a-4a8d-ad89-89ac9c7e78f6 None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:17.482533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b1e98e2a-f69a-4a8d-ad89-89ac9c7e78f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.539945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 844/1662] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:17.569503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2cef-99e1-4479-9298-75a5c7851b6b None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.570115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2cef-99e1-4479-9298-75a5c7851b6b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.570594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c2cef-99e1-4479-9298-75a5c7851b6b None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.573690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 819/1663] 10.222.0.18 () {54 vars in 961 bytes} [Tue Sep 13 19:50:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:17.584345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.585040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.585531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.644541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:17.650166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.692224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2cef-99e1-4479-9298-75a5c7851b6b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.701860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 845/1664] 10.222.0.18 () {58 vars in 1058 bytes} [Tue Sep 13 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:17.714197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c2cef-99e1-4479-9298-75a5c7851b6b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.730410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.731485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.733333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.733878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.756002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.761912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.805042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 820/1665] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:20.172513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:20.327139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfaa8e4-8e06-4431-9ae2-483720c26f8d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:20.344269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:20.345268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:20.345714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:20.346156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] PATH_INFO: `/v3/users/0f550b80005846fd8e2f865962207456` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:20.347661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f550b80005846fd8e2f865962207456)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:20.348964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: WARNING py.warnings [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:20.348964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: warnings.warn(msg) Sep 13 19:50:20.348964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: Sep 13 19:50:20.350506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:20.551307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f550b80005846fd8e2f865962207456'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.568506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f550b80005846fd8e2f865962207456'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.569984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f550b80005846fd8e2f865962207456'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.571348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f550b80005846fd8e2f865962207456'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.605222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f550b80005846fd8e2f865962207456'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:20.672675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 846/1666] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:20 2022] DELETE /identity/v3/users/0f550b80005846fd8e2f865962207456 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:22.477927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:22.497610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e404de-1bbd-4d24-ba16-b7a32a3e1707 service placement] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:22.655013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:22.658455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:22.665983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:22.668533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.670355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.670651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] PATH_INFO: `/v3/users/abbec9edbc8741dfa4d47f60c64fc0f5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.674863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbec9edbc8741dfa4d47f60c64fc0f5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:22.677053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:22.699464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb1aff-a180-474c-af41-b4d57b4df5e6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:22.715287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:22.716668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.717713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.718850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] PATH_INFO: `/v3/projects/7774072241f54acf8a9b444a58075b02` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.737256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7774072241f54acf8a9b444a58075b02)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:22.741521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:22.761267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:22.854632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:22.882824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbec9edbc8741dfa4d47f60c64fc0f5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.901055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbec9edbc8741dfa4d47f60c64fc0f5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.902424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbec9edbc8741dfa4d47f60c64fc0f5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.903492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbec9edbc8741dfa4d47f60c64fc0f5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.910622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7774072241f54acf8a9b444a58075b02', 'request_id': 'req-56829569-163d-4128-b1f5-b5d4defb3982'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.932061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7774072241f54acf8a9b444a58075b02', 'request_id': 'req-56829569-163d-4128-b1f5-b5d4defb3982'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.954628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1b6e7a99-31b4-4db0-8531-5043306ddb05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbec9edbc8741dfa4d47f60c64fc0f5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.962254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7774072241f54acf8a9b444a58075b02', 'request_id': 'req-56829569-163d-4128-b1f5-b5d4defb3982'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.996410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 821/1667] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:22 2022] DELETE /identity/v3/users/abbec9edbc8741dfa4d47f60c64fc0f5 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:23.000861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.002103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.002520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.003752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] The token cache is being invalidate because project 7774072241f54acf8a9b444a58075b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:23.005345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.012183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-56829569-163d-4128-b1f5-b5d4defb3982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:23.050224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 847/1668] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:22 2022] DELETE /identity/v3/projects/7774072241f54acf8a9b444a58075b02 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:23.056548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda684-3363-433e-b6de-4b4370ca9110 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.056814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda684-3363-433e-b6de-4b4370ca9110 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.069114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fda684-3363-433e-b6de-4b4370ca9110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.132733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b5fda684-3363-433e-b6de-4b4370ca9110 None None] MFA Rules not processed for user `dbea9a6a35c6445a9ab76ee7278abda4`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.135566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b5fda684-3363-433e-b6de-4b4370ca9110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.147679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.150327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.366701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 848/1669] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:23.368430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.371507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.372418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.392582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 822/1670] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:23.393467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.394944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.432528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:23.433869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.741362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f6692-e6c3-401b-98db-389aaba6648e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:23.786019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:23.796629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:23.798593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:23.798882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] PATH_INFO: `/v3/users/f4e468bf91564abb95f739fc00bd35d5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:23.801661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 849/1671] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:23.805929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e468bf91564abb95f739fc00bd35d5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:23.807309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:23.814056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:23.819338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.942948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:23.946494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.022990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e468bf91564abb95f739fc00bd35d5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.043866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e468bf91564abb95f739fc00bd35d5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.045473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e468bf91564abb95f739fc00bd35d5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.047893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e468bf91564abb95f739fc00bd35d5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.102754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e468bf91564abb95f739fc00bd35d5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.106497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cf62-317c-419e-bf08-34f372e09b1f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.129642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.130291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.130574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.130844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.137191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 823/1672] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:23 2022] DELETE /identity/v3/users/f4e468bf91564abb95f739fc00bd35d5 => generated 0 bytes in 743 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:24.138554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.140662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.153516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.173079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.178555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.205550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 850/1673] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:23 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:24.206546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.210308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.417508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4242452-2d1e-4f2c-bba5-5576b43e0304 service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.483664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a8479-b664-4fc7-a4f8-2c306e90cad1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.506538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.511078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.511364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.515124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.522534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.612853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.614429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.615132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.616246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.620521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.751842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.754324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5baa174-3187-4c2b-8f27-fef862fb370d service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.798702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 824/1674] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 659 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:50:24.930077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.940840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.032654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 851/1675] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 826 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:27.028620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be867-e36e-4e19-aee2-50316314a159 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.038161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be867-e36e-4e19-aee2-50316314a159 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.038161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394be867-e36e-4e19-aee2-50316314a159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.133666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-394be867-e36e-4e19-aee2-50316314a159 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:27.140410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-394be867-e36e-4e19-aee2-50316314a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.166469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 825/1676] 10.222.0.26 () {58 vars in 1015 bytes} [Tue Sep 13 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:27.300804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.314569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8123df4-7ecc-4650-8307-eb495825644d service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.330197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.331127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.333332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.335898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.380236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.381801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf926fa-677b-4787-923c-79fad8a8f7b1 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.428586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 852/1677] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:27.714002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394be867-e36e-4e19-aee2-50316314a159 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.715542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-394be867-e36e-4e19-aee2-50316314a159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:27.790415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394be867-e36e-4e19-aee2-50316314a159 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.824270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.824992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.829545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.829931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] PATH_INFO: `/v3/projects/9b65eaca28a840cea18272a521f5c91b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.833080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b65eaca28a840cea18272a521f5c91b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.834936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: WARNING py.warnings [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:27.834936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: warnings.warn(msg) Sep 13 19:50:27.834936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: Sep 13 19:50:27.837471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.897484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:28.021491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b65eaca28a840cea18272a521f5c91b', 'request_id': 'req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.062323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b65eaca28a840cea18272a521f5c91b', 'request_id': 'req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.085772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b65eaca28a840cea18272a521f5c91b', 'request_id': 'req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.124630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] The token cache is being invalidate because project 9b65eaca28a840cea18272a521f5c91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:28.127315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.134144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.143674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 826/1678] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/projects/9b65eaca28a840cea18272a521f5c91b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:28.147036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582474-af42-4d95-93c2-4b323e0812ce None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.153350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582474-af42-4d95-93c2-4b323e0812ce None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.153350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98582474-af42-4d95-93c2-4b323e0812ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.167093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:28.212708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-98582474-af42-4d95-93c2-4b323e0812ce None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.230741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-98582474-af42-4d95-93c2-4b323e0812ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.275552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.277411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.411775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb75d09-00e3-40e0-bbda-3ed3f40fdd77 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.417670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.418457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.418748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.419017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] PATH_INFO: `/v3/projects/c23109b34b3c4f75ae636ba98d7b8396` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.420215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23109b34b3c4f75ae636ba98d7b8396)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.423175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.500530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 853/1679] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.505873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.506646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.508359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.548679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23109b34b3c4f75ae636ba98d7b8396', 'request_id': 'req-5ae79071-ae4e-4cd4-886a-af1d23e506ed'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.576068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23109b34b3c4f75ae636ba98d7b8396', 'request_id': 'req-5ae79071-ae4e-4cd4-886a-af1d23e506ed'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.579306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.588107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.609360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23109b34b3c4f75ae636ba98d7b8396', 'request_id': 'req-5ae79071-ae4e-4cd4-886a-af1d23e506ed'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.660105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] The token cache is being invalidate because project c23109b34b3c4f75ae636ba98d7b8396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:28.660354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.662517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5ae79071-ae4e-4cd4-886a-af1d23e506ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:28.678443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 827/1680] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:28 2022] DELETE /identity/v3/projects/c23109b34b3c4f75ae636ba98d7b8396 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:28.681058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63053d-e734-44db-a2c6-7a558a53cf87 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.684776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63053d-e734-44db-a2c6-7a558a53cf87 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.685306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b63053d-e734-44db-a2c6-7a558a53cf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.761879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8b63053d-e734-44db-a2c6-7a558a53cf87 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.763200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8b63053d-e734-44db-a2c6-7a558a53cf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.843006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:28.884012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 854/1681] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.885326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.901648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feff81d-f920-4736-88e9-6b8ad819254a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.921736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.922705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.923177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.923606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.924432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:28.924917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.929828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.931925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 828/1682] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.940256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.940762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.941446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.970691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 855/1683] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:28.971923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.994159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:28.996354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.997049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac768733-f740-4f8e-b30f-8a41d650bd9d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.013577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.014668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.015093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.015518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.016323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.020175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 829/1684] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:29.020622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.027845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.046556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205949ce-4dc9-4d49-8790-f35e6684571a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.068115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.068920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.069341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.069862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.070732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:29.071218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.073374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.104854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 830/1685] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:50:29.111149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.121034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 856/1686] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:28 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:29.122042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.128025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4ca6-06e9-4193-bbd5-676011b4d3c3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.273227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.274548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.275068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.275633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.276688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.279430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.342567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 831/1687] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:29 2022] POST /identity/v3/projects => generated 403 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:29.347469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.372363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd133351-0bc0-44ee-bd1b-3dfeff7c8aef admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.388525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:29.569294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.574525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.576774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.578990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.581650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.594420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.626179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03142e-7583-48d7-9e42-9fa51acbd646 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.653885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.661370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.663041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.665218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.666213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.673592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.696182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 857/1688] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:29 2022] POST /identity/v3/users => generated 290 bytes in 574 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:29.706045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.766729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 832/1689] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:29 2022] POST /identity/v3/users => generated 302 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:29.769168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3a0f1-540c-44b5-ba0f-b1f5c710dac4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.770344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.793375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.794210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.794632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.795100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.799830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.801540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1797ff4-9936-4e12-b15d-9fe7072bfa18 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.804120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.830963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.833097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.833514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.833727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.834447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.836305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.850620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 858/1690] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:29.858113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.858872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 833/1691] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:29.861480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.875378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705c34bf-0e1a-4a7b-9ff1-b122fb447603 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.884454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.888146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.889079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.890256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.890969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.895013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e864c4c1-c1a8-4661-9e42-3e568ca6da24 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.901096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.919895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.920767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.931296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.932159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.938393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.942230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.961392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 834/1692] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:29.962581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.983312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 859/1693] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:29.986920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:30.010840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f029de-27a8-4fef-8b0c-a18b8fac211f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:30.014141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c7469-b483-4501-820f-9f2b8d53acea admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:30.020473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:30.021082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.021527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.021844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] PATH_INFO: `/v3/projects/d63fc47cba2e439d96e19dee9f26b90c/users/fc811da9961649b98ee93638c436e2ad/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.048469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:30.053208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.054103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.056226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] PATH_INFO: `/v3/projects/79d0d28d30e84014bf30b248381ce8c6/users/e91f0d2561c84f8c92abed0deddbd5cc/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.065054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:30.067418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63fc47cba2e439d96e19dee9f26b90c, user_id=fc811da9961649b98ee93638c436e2ad, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:30.071011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e32f41-ceef-4bcc-b7d1-325acf8a3636 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:30.093510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 860/1694] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:29 2022] PUT /identity/v3/projects/d63fc47cba2e439d96e19dee9f26b90c/users/fc811da9961649b98ee93638c436e2ad/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:30.103733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb853bf-57e2-444a-b39d-3ca24c5ec9a3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.103967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d0d28d30e84014bf30b248381ce8c6, user_id=e91f0d2561c84f8c92abed0deddbd5cc, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:30.104578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb853bf-57e2-444a-b39d-3ca24c5ec9a3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.104578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb853bf-57e2-444a-b39d-3ca24c5ec9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.110524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941292b-9caa-4ae2-9f07-438c9ef98535 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:30.160444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 835/1695] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:29 2022] PUT /identity/v3/projects/79d0d28d30e84014bf30b248381ce8c6/users/e91f0d2561c84f8c92abed0deddbd5cc/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:30.164789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.165405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.165685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.166369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8bb853bf-57e2-444a-b39d-3ca24c5ec9a3 None None] MFA Rules not processed for user `fc811da9961649b98ee93638c436e2ad`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:30.168075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8bb853bf-57e2-444a-b39d-3ca24c5ec9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:30.268540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] MFA Rules not processed for user `e91f0d2561c84f8c92abed0deddbd5cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:30.275280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:30.377907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 861/1696] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:30.379575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.382749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.384147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.490539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] MFA Rules not processed for user `e60df169b5dc4aa59b5b0246544ef2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:30.496340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:30.537296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 836/1697] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:30.537296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:30.720691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:30.809703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 862/1698] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:30.810978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:30.844042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19bf15-8028-4a80-bd97-734b2d324e9b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:30.848461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275fa3d-b763-466a-9576-1cd667caeb9b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:30.871944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:30.873113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.873400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.873770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.877704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:30.878597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:30.884490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:30.887464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.887775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.888053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.888755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:30.906752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.036506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 837/1699] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/projects => generated 379 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:31.041083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.047092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.075941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 863/1700] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/projects => generated 405 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:31.090613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.348337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:31.502592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0dff-53e3-4c03-a697-6da71ce41c1f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.555338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb35e5-2b9e-4cdb-8f36-8de1caf6ea70 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.591863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.592736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.593219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.596383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.597428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.601640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.671939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 864/1701] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/users => generated 296 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:31.673257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.697213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fc8732-2351-43b6-8358-477c016d5814 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.717357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.718582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.719076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.719559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.720429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.722717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.792483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.792483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.792483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.792483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.793130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 865/1702] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/users => generated 309 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:31.794137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.817723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc608ff5-20f5-4bb7-abf9-8b70ba586d89 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.842251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.842251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.842251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.843590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.846357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.848347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.870773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 866/1703] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:31.873359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.913438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163d58a-7c80-4b4a-af03-a2714ddb9734 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.930914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.931764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.941733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.942460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.943308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.948012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.977984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.980129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.983702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 867/1704] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:31.987503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.003609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 838/1705] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 967 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:32.005034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.007486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a38ae6-2987-4f60-883a-2f82a5c6a388 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.022697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67cd71-74c2-48ce-909e-90b08636850d service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.023726 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.024593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.024850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.025301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] PATH_INFO: `/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87/users/9adc0fae013e45bc8c3554592c14b588/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.042076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.043020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.043445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.043834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] PATH_INFO: `/v3/projects/4565cfa27c634c328bf5110be24d6a18/users/740fe0b3e76b4f9c832ff2f5082b3ba4/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.068553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44ef1642a204d1c9cc3a849d84c7c87, user_id=9adc0fae013e45bc8c3554592c14b588, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.072627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0cfcb-2f4b-4bcc-8029-982b5ab1d8d2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.091801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4565cfa27c634c328bf5110be24d6a18, user_id=740fe0b3e76b4f9c832ff2f5082b3ba4, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.094896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.126913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 868/1706] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:31 2022] PUT /identity/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87/users/9adc0fae013e45bc8c3554592c14b588/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:32.132162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.132162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.132162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.175726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 839/1707] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:32 2022] PUT /identity/v3/projects/4565cfa27c634c328bf5110be24d6a18/users/740fe0b3e76b4f9c832ff2f5082b3ba4/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:32.177625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.278234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] MFA Rules not processed for user `a1c9d4af0a98406692e1d79293844c68`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:32.279829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:32.426032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:32.452065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450846e9-aec1-4bc8-818f-57c435cd6a82 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.479976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.483446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.485395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.486871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.487736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.494867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.557723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 840/1708] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:32.570906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.596640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 869/1709] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:32.600657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.639876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:32.652017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11cb21-1d78-4c8a-aa4c-ba86714cd189 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.673552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.673928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cca2bf-cc8c-4295-97cf-08e465c4e1bd None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.675704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.679165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.680591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.682203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.686257 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a33e-d3aa-4acd-aaf0-daaa7ee3f968 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.693807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.695438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.695438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.696845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] PATH_INFO: `/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87/users/9adc0fae013e45bc8c3554592c14b588/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.712106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e44ef1642a204d1c9cc3a849d84c7c87, user_id=9adc0fae013e45bc8c3554592c14b588, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.721810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.732180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 841/1710] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:32.735522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c2ab-6fdf-4cea-9f44-c50e08e409d5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.736667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c2ab-6fdf-4cea-9f44-c50e08e409d5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.737820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f46c2ab-6fdf-4cea-9f44-c50e08e409d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.791887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 870/1711] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:32 2022] PUT /identity/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87/users/9adc0fae013e45bc8c3554592c14b588/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:32.793810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.889945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9f46c2ab-6fdf-4cea-9f44-c50e08e409d5 None None] MFA Rules not processed for user `a1c9d4af0a98406692e1d79293844c68`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:32.892017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9f46c2ab-6fdf-4cea-9f44-c50e08e409d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.077661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c696e0-2f12-4f06-a94e-9a9c7ca0ba9c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:33.099764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:33.100472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.105617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.106005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] PATH_INFO: `/v3/projects/4565cfa27c634c328bf5110be24d6a18/users/740fe0b3e76b4f9c832ff2f5082b3ba4/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.114672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4565cfa27c634c328bf5110be24d6a18, user_id=740fe0b3e76b4f9c832ff2f5082b3ba4, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:33.128232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86a3edd-32f5-4342-b539-dc43d082815c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:33.186469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 842/1712] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 453 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:33.188420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 871/1713] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:32 2022] PUT /identity/v3/projects/4565cfa27c634c328bf5110be24d6a18/users/740fe0b3e76b4f9c832ff2f5082b3ba4/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:33.199364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.200310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.202522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.203511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44089b-761f-4fed-b138-e10d2268ae22 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.207034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44089b-761f-4fed-b138-e10d2268ae22 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.207611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44089b-761f-4fed-b138-e10d2268ae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.359169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] MFA Rules not processed for user `dbea9a6a35c6445a9ab76ee7278abda4`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:33.378705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.454608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dc44089b-761f-4fed-b138-e10d2268ae22 None None] MFA Rules not processed for user `9adc0fae013e45bc8c3554592c14b588`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:33.470993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dc44089b-761f-4fed-b138-e10d2268ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.706645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 872/1714] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 517 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:33.710902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:33.715849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.916829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 843/1715] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 730 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:33.920260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230a02-09a7-468b-82fa-249469974a30 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.921623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230a02-09a7-468b-82fa-249469974a30 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.924324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22230a02-09a7-468b-82fa-249469974a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.970960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:34.009900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-22230a02-09a7-468b-82fa-249469974a30 None None] MFA Rules not processed for user `740fe0b3e76b4f9c832ff2f5082b3ba4`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:34.011539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-22230a02-09a7-468b-82fa-249469974a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.013971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9da288-ecd4-49b9-a53f-e2e947ed75d0 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.043337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.043979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.044267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.044534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.239485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.241500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1284ba-22c2-4453-910e-467079ef5338 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.272358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 844/1716] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:34.276358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.276788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.278408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.286653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 845/1717] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:50:34 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:34.286653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.288427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.293602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 873/1718] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:34.298845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.307181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.307768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.321172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:34.390314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:34.395892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.438103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 874/1719] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:34.439401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.441657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.482585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e8c69f-97d2-4943-9b8c-707850532172 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.490564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626f428-682a-44f0-b800-c7022b6a28e3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.495166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.496149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.496661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.497191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.501759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.502600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.502887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.503156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.535682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.538403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.540027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.541908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.558097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 875/1720] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:34.578420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 846/1721] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:34.838771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.849310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75693335-5bb9-4484-9810-8e06ea3a2e65 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.860573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.861348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.862469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.862887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.009228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:35.011658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35110f4-9064-4038-b0be-c0e531b1653b service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:35.025055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 876/1722] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:35.714428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:35.929027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6aad32-ffff-4202-904e-aaf0296f5575 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:35.959346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:35.960917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.961303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.961680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] PATH_INFO: `/v3/users/e91f0d2561c84f8c92abed0deddbd5cc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.963122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e91f0d2561c84f8c92abed0deddbd5cc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:35.966052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:36.309289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e91f0d2561c84f8c92abed0deddbd5cc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:36.331248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e91f0d2561c84f8c92abed0deddbd5cc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:36.332502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e91f0d2561c84f8c92abed0deddbd5cc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:36.333574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e91f0d2561c84f8c92abed0deddbd5cc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:36.430446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e91f0d2561c84f8c92abed0deddbd5cc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:36.510442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 847/1723] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:35 2022] DELETE /identity/v3/users/e91f0d2561c84f8c92abed0deddbd5cc => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:36.515438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cba8c-853c-4966-b19e-2f0aed800f40 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:36.516049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cba8c-853c-4966-b19e-2f0aed800f40 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:36.516049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26cba8c-853c-4966-b19e-2f0aed800f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:36.557184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a26cba8c-853c-4966-b19e-2f0aed800f40 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:36.561301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a26cba8c-853c-4966-b19e-2f0aed800f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:36.774829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 877/1724] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:36.835681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:36.841215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:37.007996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae0ee-24fd-4a40-919a-e4f5dbb046ba None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.008663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae0ee-24fd-4a40-919a-e4f5dbb046ba None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.009207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cae0ee-24fd-4a40-919a-e4f5dbb046ba None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.010926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 878/1725] 10.222.0.18 () {54 vars in 912 bytes} [Tue Sep 13 19:50:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:50:37.036340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.037208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.037727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.059430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:37.093348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:37.095031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:37.118370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:37.243924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 879/1726] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:37.251168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c34478-b647-418f-9770-1792cb7a5826 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:37.272687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:37.273815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.274239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.275027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.300731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:37.314842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e338e3-d13b-49bf-b672-7c0e05ea8cb6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:37.326798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:37.327831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.328407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.329080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.335139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:37.338343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:37.357295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:37.367564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:37.379250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 848/1727] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:37.417998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 880/1728] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:38.611310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:38.613000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:38.848027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:38.860235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:38.982679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412a7ab-0b96-41b1-8dd5-ef207e8dcf68 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:38.992294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6e2d19-317f-4c30-837c-4aea5b7fcb38 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:39.028132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:39.034532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:39.038707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:39.039260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:39.039727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:39.044384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.047023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:39.047575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:39.048075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] PATH_INFO: `/v3/users/f3bb91dc3f1347e6acd722b0af28ca32` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:39.084171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3bb91dc3f1347e6acd722b0af28ca32)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:39.092105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:39.098044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:39.098980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:39.132174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 849/1729] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:39.186742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:39.189212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.247957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb69e5-9417-4c08-8dda-d938e661123b service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:39.276672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:39.277406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:39.277690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:39.284340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] PATH_INFO: `/v3/projects/79d0d28d30e84014bf30b248381ce8c6` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:39.290686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d0d28d30e84014bf30b248381ce8c6)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:39.298255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:39.502629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3bb91dc3f1347e6acd722b0af28ca32'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.511974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d0d28d30e84014bf30b248381ce8c6', 'request_id': 'req-3a39c19b-fbad-4402-ae8c-9f8f698041b2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.542596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d0d28d30e84014bf30b248381ce8c6', 'request_id': 'req-3a39c19b-fbad-4402-ae8c-9f8f698041b2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.544717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3bb91dc3f1347e6acd722b0af28ca32'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.549542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3bb91dc3f1347e6acd722b0af28ca32'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.551521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3bb91dc3f1347e6acd722b0af28ca32'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.581543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d0d28d30e84014bf30b248381ce8c6', 'request_id': 'req-3a39c19b-fbad-4402-ae8c-9f8f698041b2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.605628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2a80a360-2935-447e-a196-3ef95074151c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3bb91dc3f1347e6acd722b0af28ca32'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.606051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] The token cache is being invalidate because project 79d0d28d30e84014bf30b248381ce8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:39.606529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.611527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:39.618927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 850/1730] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:39 2022] DELETE /identity/v3/projects/79d0d28d30e84014bf30b248381ce8c6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:39.623112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:39.626635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.671249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 881/1731] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:38 2022] DELETE /identity/v3/users/f3bb91dc3f1347e6acd722b0af28ca32 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:39.676650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:39.677712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:39.678631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:39.732529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:39.733977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.823860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:39.907120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39c19b-fbad-4402-ae8c-9f8f698041b2 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:40.031311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 882/1732] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:40.148808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:40.150687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:40.219497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:40.220288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:40.220624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:40.220928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] PATH_INFO: `/v3/users/740fe0b3e76b4f9c832ff2f5082b3ba4` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:40.226350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740fe0b3e76b4f9c832ff2f5082b3ba4)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:40.228471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:40.237872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:40.343282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f75ab76-da88-481b-8a56-0932c067d100 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:40.380450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:40.385320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:40.386368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:40.387745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:40.423198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740fe0b3e76b4f9c832ff2f5082b3ba4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.430950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:40.433563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c3e9-dd3a-41c3-9eb0-e2392be04507 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:40.451950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740fe0b3e76b4f9c832ff2f5082b3ba4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.453217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740fe0b3e76b4f9c832ff2f5082b3ba4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.456681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740fe0b3e76b4f9c832ff2f5082b3ba4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.479982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 883/1733] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:40.578124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740fe0b3e76b4f9c832ff2f5082b3ba4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.681401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 851/1734] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:39 2022] DELETE /identity/v3/users/740fe0b3e76b4f9c832ff2f5082b3ba4 => generated 0 bytes in 1059 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:41.060136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548815d-ae90-4d74-9eff-99326121773f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.061268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548815d-ae90-4d74-9eff-99326121773f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.061777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548815d-ae90-4d74-9eff-99326121773f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.142493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5548815d-ae90-4d74-9eff-99326121773f None None] MFA Rules not processed for user `a1c9d4af0a98406692e1d79293844c68`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:41.146487 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5548815d-ae90-4d74-9eff-99326121773f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.328696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 884/1735] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:41.588441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:41.590473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.766420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.769374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.769887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.828931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a90db-bced-43ec-a1ba-c6e7985a2ef7 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:41.834626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] MFA Rules not processed for user `e60df169b5dc4aa59b5b0246544ef2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:41.837495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.843530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:41.847089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.849492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.850185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] PATH_INFO: `/v3/projects/4565cfa27c634c328bf5110be24d6a18` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.862394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4565cfa27c634c328bf5110be24d6a18)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:41.865952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:41.901427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:42.009093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4565cfa27c634c328bf5110be24d6a18', 'request_id': 'req-60e036ac-f374-4472-8947-cf449722dcfc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.024326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 885/1736] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:42.036112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4565cfa27c634c328bf5110be24d6a18', 'request_id': 'req-60e036ac-f374-4472-8947-cf449722dcfc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.067266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4565cfa27c634c328bf5110be24d6a18', 'request_id': 'req-60e036ac-f374-4472-8947-cf449722dcfc'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.093374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] The token cache is being invalidate because project 4565cfa27c634c328bf5110be24d6a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:42.094063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.097726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60e036ac-f374-4472-8947-cf449722dcfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.098548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.100250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.100381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 852/1737] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:41 2022] DELETE /identity/v3/projects/4565cfa27c634c328bf5110be24d6a18 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:42.105368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09dab-6a93-4a37-83c9-3b2f57bc780c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.105649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09dab-6a93-4a37-83c9-3b2f57bc780c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.106096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c09dab-6a93-4a37-83c9-3b2f57bc780c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.133040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231867e-1b91-40d5-939f-23964092cc63 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:42.158683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c2c09dab-6a93-4a37-83c9-3b2f57bc780c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:42.164006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c2c09dab-6a93-4a37-83c9-3b2f57bc780c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.252030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 853/1738] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:42.258839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:42.259808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.260270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.260701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] PATH_INFO: `/v3/projects/261dad42b0bc43d6832b77f5850f18d9` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.260854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.261334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.261666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.262937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] RBAC: Authorizing `identity:delete_project(project_id=261dad42b0bc43d6832b77f5850f18d9)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:42.265219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:42.286625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:42.297191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:42.298365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.310557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 854/1739] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:42.325979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.345461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566b52b-ba4d-4e48-9f62-e39f506d3c3f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:42.392334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '261dad42b0bc43d6832b77f5850f18d9', 'request_id': 'req-983622f2-8366-44c9-94d8-e0b0b7678196'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.423590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '261dad42b0bc43d6832b77f5850f18d9', 'request_id': 'req-983622f2-8366-44c9-94d8-e0b0b7678196'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.460682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '261dad42b0bc43d6832b77f5850f18d9', 'request_id': 'req-983622f2-8366-44c9-94d8-e0b0b7678196'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.509248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] The token cache is being invalidate because project 261dad42b0bc43d6832b77f5850f18d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:42.509760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.512427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.520406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 886/1740] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:42 2022] DELETE /identity/v3/projects/261dad42b0bc43d6832b77f5850f18d9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:42.520717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:42.520717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.520717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.520717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.529153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:42.529359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.529448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:42.529613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.533984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:42.595672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983622f2-8366-44c9-94d8-e0b0b7678196 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:42.606933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 855/1741] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:42.612735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.616722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:42.618751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.620658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.621207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.623229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] PATH_INFO: `/v3/users/2362d34ebd73470f9515503e081dcdfa` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.629965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2362d34ebd73470f9515503e081dcdfa)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:42.638543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:42.679338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f519b79-f2e3-4690-8d92-02dce5a99250 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:42.687675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:42.688796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.693792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.694876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.696487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:42.705606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:42.810278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 856/1742] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:42 2022] POST /identity/v3/projects => generated 373 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:42.822919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.841235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:42.848027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2362d34ebd73470f9515503e081dcdfa'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.886460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2362d34ebd73470f9515503e081dcdfa'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.888393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2362d34ebd73470f9515503e081dcdfa'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.889944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2362d34ebd73470f9515503e081dcdfa'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:42.938039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2362d34ebd73470f9515503e081dcdfa'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.064718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b36aa8-7a38-4586-887b-71bf2edc93f6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:43.071369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 887/1743] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:42 2022] DELETE /identity/v3/users/2362d34ebd73470f9515503e081dcdfa => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:43.277774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:43.278413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.278682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.279005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.279754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:43.281717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.332272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 857/1744] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:42 2022] POST /identity/v3/users => generated 287 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:43.336991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:43.341242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:43.378211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3bfded-f094-4d7b-ad74-5b1848078b12 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:43.393420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:43.394073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.394374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.394608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.395348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:43.398484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.461353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 888/1745] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:43.463309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:43.487929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03edde19-774f-43df-9ac0-72605adc5f46 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:43.499474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:43.501368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.501566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.501860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.503139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:43.505182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d23a21-6205-41c6-a084-82da99b50fc2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.533542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 858/1746] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:43.538439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:43.572465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65bb48-00d6-4339-b23b-5a62977bb1e2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:43.582725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:43.584625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.584919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.585200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] PATH_INFO: `/v3/projects/a1fa0f643add4604ab0ee6d138042aab/users/001b63d9c6d74d6896186040266f02af/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.640711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1fa0f643add4604ab0ee6d138042aab, user_id=001b63d9c6d74d6896186040266f02af, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:43.648719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.677370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 889/1747] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:43 2022] PUT /identity/v3/projects/a1fa0f643add4604ab0ee6d138042aab/users/001b63d9c6d74d6896186040266f02af/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:43.689670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.689960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.690280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.790681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:43.793234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:43.796234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] MFA Rules not processed for user `001b63d9c6d74d6896186040266f02af`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:43.797505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:43.966094 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:44.037764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf1e9e-e6df-4054-b1b3-11eae1cf1ee6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.046205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.047722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.047722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.047722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] PATH_INFO: `/v3/projects/44627e8020284298ba49f1e82b589e79` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.048828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44627e8020284298ba49f1e82b589e79)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.051033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.114922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 859/1748] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:44.117678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.147070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43e93f-41d9-41b9-bb5e-2c952119a6ac None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.165184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44627e8020284298ba49f1e82b589e79', 'request_id': 'req-741c39c1-d65f-47da-a2c5-44e3a3d14d09'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:44.190408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44627e8020284298ba49f1e82b589e79', 'request_id': 'req-741c39c1-d65f-47da-a2c5-44e3a3d14d09'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:44.200842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:44.207389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44627e8020284298ba49f1e82b589e79', 'request_id': 'req-741c39c1-d65f-47da-a2c5-44e3a3d14d09'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:44.237454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] The token cache is being invalidate because project 44627e8020284298ba49f1e82b589e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:44.237844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:44.238882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-741c39c1-d65f-47da-a2c5-44e3a3d14d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:44.245714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 890/1749] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:43 2022] DELETE /identity/v3/projects/44627e8020284298ba49f1e82b589e79 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:44.247792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99c6e-e450-43d8-a5da-bf1ae593b9ec None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.248072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99c6e-e450-43d8-a5da-bf1ae593b9ec None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.248344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc99c6e-e450-43d8-a5da-bf1ae593b9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.298933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7dc99c6e-e450-43d8-a5da-bf1ae593b9ec None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:44.301764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7dc99c6e-e450-43d8-a5da-bf1ae593b9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.309911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.311344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.311615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.311887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.312532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.317265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.322500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 891/1750] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:44.331387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.332567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.333026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.375684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:44.380518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.388848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 860/1751] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:44.392638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.397862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.475948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 892/1752] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:44.475948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.487060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3523a-243f-4352-a832-648f20e635ab None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.492667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.493954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.493954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.494602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.495121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:44.495539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.497661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.523718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 893/1753] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:44.525254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.540430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00246a75-cf58-4b53-aee4-8bedc4aa4be6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.548179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.548948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.549229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.549515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.550245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.553663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.608791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 894/1754] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:44.609893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.617213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672eb38-c426-4320-8b5e-f0e623ba91dd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.675194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:44.745761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.747587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.747937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.748314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.750115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.755973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ede9c-15a3-482d-91c9-43a55ff8165c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.761949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.780984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.782835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.784452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.784873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.788862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.793056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.802371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 861/1755] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/users => generated 292 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:44.806371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.827563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9323d9a-92a8-407e-9775-83cdc902f90f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.831603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 895/1756] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:44.836187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.837367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.837367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.837597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.838001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.839383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.840506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.853681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 862/1757] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:44.857561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df3184e-dd8e-4eda-be75-5193004f7c44 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.858737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.867458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.868086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.868360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.868625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.869330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.870938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.884940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755c02-28bc-49bc-be49-e98cde611452 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.886999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 896/1758] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:44.898871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.901545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.902380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.902838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.903145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] PATH_INFO: `/v3/projects/215388874256426d96f64c4f380393e1/users/6867cc72e0ff4b73a8c503a9743cd2df/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.909667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d21c9-650b-4e1d-a2de-039b4c47ddca admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.914789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.925033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.925033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.925033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.925033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.925033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.925468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=215388874256426d96f64c4f380393e1, user_id=6867cc72e0ff4b73a8c503a9743cd2df, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.931671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:44.931828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 897/1759] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:44.936230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.949637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 863/1760] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:44 2022] PUT /identity/v3/projects/215388874256426d96f64c4f380393e1/users/6867cc72e0ff4b73a8c503a9743cd2df/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:44.950882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c1e86-cdb1-4673-8a1d-109534616b40 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.954229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.967373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.056930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.058134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.058415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.058866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] PATH_INFO: `/v3/projects/ef19fef6f0aa4a888c4655d9652c5879/users/0f390842e1884ec8ab1e0978e25cecf5/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.080693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37f9581-7612-4d21-ba32-02550f970509 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.090031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.090734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.091076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.091413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.092148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.094003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.099237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef19fef6f0aa4a888c4655d9652c5879, user_id=0f390842e1884ec8ab1e0978e25cecf5, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.102282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426faed-7afb-4006-97db-ee4570ab7369 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.115740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 864/1761] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:44 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:45.116808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.121060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 898/1762] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:44 2022] PUT /identity/v3/projects/ef19fef6f0aa4a888c4655d9652c5879/users/0f390842e1884ec8ab1e0978e25cecf5/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:45.127954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb0666-c4fb-4e33-8386-44ba82990797 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.131238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.131590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.131770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.169108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] MFA Rules not processed for user `0f390842e1884ec8ab1e0978e25cecf5`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:45.170921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.197920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.198609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.198894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.199163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] PATH_INFO: `/v3/projects/215388874256426d96f64c4f380393e1/users/6867cc72e0ff4b73a8c503a9743cd2df/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.201201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=215388874256426d96f64c4f380393e1, user_id=6867cc72e0ff4b73a8c503a9743cd2df, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.203971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bfdbb-d37d-4494-b718-1f19dc9814ac admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.216329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 865/1763] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:45 2022] PUT /identity/v3/projects/215388874256426d96f64c4f380393e1/users/6867cc72e0ff4b73a8c503a9743cd2df/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:45.227090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.227373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.227654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.255983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 899/1764] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:45.260324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] MFA Rules not processed for user `6867cc72e0ff4b73a8c503a9743cd2df`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:45.261684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.262006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.273543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b03cc9-9660-4ae9-a1dd-49b50bc27e3e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.282349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.343605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.344205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.344484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.344580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 866/1765] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:45.344830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.345698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.347766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.357456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.358947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.385092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 900/1766] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/projects => generated 383 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:45.389956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.399489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.457325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a193197-59fa-4f76-9b9d-2927049bf779 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.477754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89adba0d-d6e1-45fd-9145-cbca1fdaab54 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.494138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.494777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.495054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.495318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.495965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.498461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.533098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 901/1767] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:45.537271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.539300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.539953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.540230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.540499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.549571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beab03-d936-4572-beec-b7b6f3ee4f11 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.555800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.556922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.556922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.557148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.557733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.559308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.576304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 902/1768] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:45.582206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.598282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847de85-0ee2-41da-8eaf-52c58d441311 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.603805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.604623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.605022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.605432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] PATH_INFO: `/v3/projects/2dcfcd86b1094151a9d77f23929112e9/users/2d65017ac65a428c8a5b4a379661011a/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.635733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcfcd86b1094151a9d77f23929112e9, user_id=2d65017ac65a428c8a5b4a379661011a, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.638680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.657129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 903/1769] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:45 2022] PUT /identity/v3/projects/2dcfcd86b1094151a9d77f23929112e9/users/2d65017ac65a428c8a5b4a379661011a/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:45.661294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.662428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.666023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.680266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 867/1770] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:45.771646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.812036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5ca93-be90-45da-8b9b-c73e0ca594d7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.828246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.828246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.830035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.830489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.832848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.839439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee6df5-68aa-4861-b573-dafa20be4944 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.880989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 904/1771] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:45 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:45.890922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.903363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73731628-7463-4ca6-bc24-46210fe4feb6 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:45.909120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:45.910092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.910492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.910756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] PATH_INFO: `/v3/projects/2dcfcd86b1094151a9d77f23929112e9/users/2d65017ac65a428c8a5b4a379661011a/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.912844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dcfcd86b1094151a9d77f23929112e9, user_id=2d65017ac65a428c8a5b4a379661011a, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:45.916138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.929697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 868/1772] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:45 2022] PUT /identity/v3/projects/2dcfcd86b1094151a9d77f23929112e9/users/2d65017ac65a428c8a5b4a379661011a/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:45.947000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.947311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.947675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.995221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] MFA Rules not processed for user `2d65017ac65a428c8a5b4a379661011a`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:45.997000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:46.189419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 905/1773] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:46.210879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:46.306606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d10e3d4-d842-4076-a153-f8789db98a3f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:46.313716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:46.314313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.314584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.314854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.326621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:46.328088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dad4c4f-4c11-4018-81cb-2eb0583cde9c service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:46.345576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 869/1774] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:46.442181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:46.530700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f62ff-b417-4df8-9b80-7fa52ec4b0b6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:46.542409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:46.543787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.544306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.544757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] PATH_INFO: `/v3/users/001b63d9c6d74d6896186040266f02af` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.548618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=001b63d9c6d74d6896186040266f02af)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:46.552305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:46.633002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001b63d9c6d74d6896186040266f02af'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:46.654835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001b63d9c6d74d6896186040266f02af'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:46.655442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001b63d9c6d74d6896186040266f02af'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:46.656848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001b63d9c6d74d6896186040266f02af'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:46.665482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001b63d9c6d74d6896186040266f02af'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:46.685188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:46.693053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 906/1775] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:46 2022] DELETE /identity/v3/users/001b63d9c6d74d6896186040266f02af => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:46.693615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.693903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.694171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.716203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:46.718225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:46.734787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:46.801715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 870/1776] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:46.816627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:46.819261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:46.912139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59724349-67e1-45e3-ac03-d6333eccbbbd admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:46.919587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:46.922202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.922652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.923096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.943227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:46.945324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1240c5-ca93-494c-b316-44af03bcb47a service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:46.957876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 907/1777] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:47.148734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.151268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.173714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3ddce-2695-4bc3-9119-20e133a80624 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.178794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.179733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.180000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.180430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] PATH_INFO: `/v3/users/0f390842e1884ec8ab1e0978e25cecf5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.181642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f390842e1884ec8ab1e0978e25cecf5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.183549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.242289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f390842e1884ec8ab1e0978e25cecf5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.255052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f390842e1884ec8ab1e0978e25cecf5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.255800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f390842e1884ec8ab1e0978e25cecf5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.256629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f390842e1884ec8ab1e0978e25cecf5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.279763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f390842e1884ec8ab1e0978e25cecf5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.323713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 871/1778] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:47 2022] DELETE /identity/v3/users/0f390842e1884ec8ab1e0978e25cecf5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:47.329076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.329613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.329748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.337900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.339505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.364763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.366032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.435037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 908/1779] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:47.445848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.447590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.463035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c10b19-0108-4915-83f0-3819ae4f19ca admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.468984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.469595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.469871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.470137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] PATH_INFO: `/v3/projects/a1fa0f643add4604ab0ee6d138042aab` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.471502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1fa0f643add4604ab0ee6d138042aab)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.474064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.530727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:47.534980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8a36c-cc1a-4fed-8943-66de8f253d85 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.554797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1fa0f643add4604ab0ee6d138042aab', 'request_id': 'req-30a27004-949a-4a25-a113-447917219d78'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.561636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1fa0f643add4604ab0ee6d138042aab', 'request_id': 'req-30a27004-949a-4a25-a113-447917219d78'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.570653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1fa0f643add4604ab0ee6d138042aab', 'request_id': 'req-30a27004-949a-4a25-a113-447917219d78'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.582082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] The token cache is being invalidate because project a1fa0f643add4604ab0ee6d138042aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:47.582368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.583025 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.584870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 872/1780] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:47 2022] DELETE /identity/v3/projects/a1fa0f643add4604ab0ee6d138042aab => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:47.588623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.590080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.601423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.602087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.602367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.602642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.604524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.610034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:47.680635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a27004-949a-4a25-a113-447917219d78 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.684015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.685537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.686431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.687107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.687387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.687662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] PATH_INFO: `/v3/users/6867cc72e0ff4b73a8c503a9743cd2df` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.689053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6867cc72e0ff4b73a8c503a9743cd2df)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.692324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.697324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 909/1781] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:47.759763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6867cc72e0ff4b73a8c503a9743cd2df'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.769603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6867cc72e0ff4b73a8c503a9743cd2df'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.770357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6867cc72e0ff4b73a8c503a9743cd2df'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.771184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6867cc72e0ff4b73a8c503a9743cd2df'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.790322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6867cc72e0ff4b73a8c503a9743cd2df'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.808828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 873/1782] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:47 2022] DELETE /identity/v3/users/6867cc72e0ff4b73a8c503a9743cd2df => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:48.119677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.122121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.162454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ec6a3-be69-481a-8d38-876d910588bc service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.254348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.256217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.258290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.258290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] PATH_INFO: `/v3/projects/ef19fef6f0aa4a888c4655d9652c5879` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.259531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef19fef6f0aa4a888c4655d9652c5879)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.262611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.323215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.326374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.343091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef19fef6f0aa4a888c4655d9652c5879', 'request_id': 'req-f94ac50b-c925-4c70-8b24-7a417753b916'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.352005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef19fef6f0aa4a888c4655d9652c5879', 'request_id': 'req-f94ac50b-c925-4c70-8b24-7a417753b916'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.378175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef19fef6f0aa4a888c4655d9652c5879', 'request_id': 'req-f94ac50b-c925-4c70-8b24-7a417753b916'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.379020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823f5b1a-fff9-4200-99f9-7abb0f5b4ef1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.398368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] The token cache is being invalidate because project ef19fef6f0aa4a888c4655d9652c5879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:48.398736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.399771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.404942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 910/1783] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:48 2022] DELETE /identity/v3/projects/ef19fef6f0aa4a888c4655d9652c5879 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:48.412395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.414248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.472572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:48.474563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:48.510371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.511405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.513246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.513808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] PATH_INFO: `/v3/projects/215388874256426d96f64c4f380393e1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.515573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=215388874256426d96f64c4f380393e1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.517626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.522795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ac50b-c925-4c70-8b24-7a417753b916 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.530537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.531516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.532008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.532514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] PATH_INFO: `/v3/users/2d65017ac65a428c8a5b4a379661011a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.534594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d65017ac65a428c8a5b4a379661011a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.536946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.616599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d65017ac65a428c8a5b4a379661011a'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.621726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '215388874256426d96f64c4f380393e1', 'request_id': 'req-4a360c1c-15e8-465f-9384-ddddb9e240c6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.631214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '215388874256426d96f64c4f380393e1', 'request_id': 'req-4a360c1c-15e8-465f-9384-ddddb9e240c6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.632114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d65017ac65a428c8a5b4a379661011a'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.633662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d65017ac65a428c8a5b4a379661011a'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.635363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d65017ac65a428c8a5b4a379661011a'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.644954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '215388874256426d96f64c4f380393e1', 'request_id': 'req-4a360c1c-15e8-465f-9384-ddddb9e240c6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.645692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f355af82-94fc-483e-ab5e-8222d3197825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d65017ac65a428c8a5b4a379661011a'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.668411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] The token cache is being invalidate because project 215388874256426d96f64c4f380393e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:48.668954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.670027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4a360c1c-15e8-465f-9384-ddddb9e240c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.675979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 874/1785] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:48 2022] DELETE /identity/v3/projects/215388874256426d96f64c4f380393e1 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:48.676314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 911/1785] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:48 2022] DELETE /identity/v3/users/2d65017ac65a428c8a5b4a379661011a => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:48.678618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.678975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.679341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.704480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.705915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.802093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 875/1786] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.814534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.814942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.815211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.863351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.865617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.893705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 912/1787] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.898758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.912305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3cf71-85be-439f-8a5c-0a7182e6f574 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.920452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.921415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.922026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.922026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.922605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:48.922883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.924687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.938299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 876/1788] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:48.943490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.953274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9d5c3-0149-47d4-aa73-d9ba828d0668 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.959049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.959790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.960105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.960365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.961167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.963690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.001289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 913/1789] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:49.004132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.022750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840087-e26a-4ca7-8069-424f2b04f2b0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.032314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.035001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.106038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:49.117148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.118699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.119305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.119985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.121154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.125270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.164535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 877/1790] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:49.166354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.166910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4125c89-3c8e-414c-b2af-116673af3d33 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.173052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.173052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.173356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.173699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] PATH_INFO: `/v3/projects/2dcfcd86b1094151a9d77f23929112e9` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.174893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dcfcd86b1094151a9d77f23929112e9)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.176834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.181503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673306a-fa57-48f0-b232-c07476599089 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.198142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.200371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.200844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.201331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.202451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.206230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.212240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:49.236939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 878/1791] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:49.239891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.254525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89bb3c6-05d6-4edc-ae3a-68cd87e71e03 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.259955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dcfcd86b1094151a9d77f23929112e9', 'request_id': 'req-ba9d4810-334e-4828-af6f-e39e9b3b7233'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:49.267635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dcfcd86b1094151a9d77f23929112e9', 'request_id': 'req-ba9d4810-334e-4828-af6f-e39e9b3b7233'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:49.281196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dcfcd86b1094151a9d77f23929112e9', 'request_id': 'req-ba9d4810-334e-4828-af6f-e39e9b3b7233'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:49.296594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] The token cache is being invalidate because project 2dcfcd86b1094151a9d77f23929112e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:49.297359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:49.298375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba9d4810-334e-4828-af6f-e39e9b3b7233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:49.300329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 914/1792] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:49 2022] DELETE /identity/v3/projects/2dcfcd86b1094151a9d77f23929112e9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:49.305572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6d79-f352-45df-a9e2-5c4772c6523f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.305957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6d79-f352-45df-a9e2-5c4772c6523f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.306131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc6d79-f352-45df-a9e2-5c4772c6523f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.336628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-93fc6d79-f352-45df-a9e2-5c4772c6523f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:49.336628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-93fc6d79-f352-45df-a9e2-5c4772c6523f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.388461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.389334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.389565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.389835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.390489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.392428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.405315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 879/1793] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.410925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.416575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.449292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6289f95a-956e-4b57-8b0d-f9f754ed7f52 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.474196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.477045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.477392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.477835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] PATH_INFO: `/v3/projects/133c15d5ab28474dbacd9935d10821f7/users/ffbbb7c1637448299beff15d42fa6c81/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.500974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 915/1794] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:49.505262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d8944-4598-469d-86f8-39fc6247b8a1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.505534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d8944-4598-469d-86f8-39fc6247b8a1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.505975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d8944-4598-469d-86f8-39fc6247b8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.537996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=133c15d5ab28474dbacd9935d10821f7, user_id=ffbbb7c1637448299beff15d42fa6c81, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.549465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.595332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e06d8944-4598-469d-86f8-39fc6247b8a1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:49.596870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e06d8944-4598-469d-86f8-39fc6247b8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.641692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 880/1795] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:49 2022] PUT /identity/v3/projects/133c15d5ab28474dbacd9935d10821f7/users/ffbbb7c1637448299beff15d42fa6c81/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:49.662222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.674778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.921813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b1ba49-9385-405c-bc18-0fbce9fde7d4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.935199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 916/1796] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:49.936834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332dc3e-a2f8-4638-8235-af5b22f7d007 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.937387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332dc3e-a2f8-4638-8235-af5b22f7d007 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.937845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332dc3e-a2f8-4638-8235-af5b22f7d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.942569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.942569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.942569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.943029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] PATH_INFO: `/v3/users/fc811da9961649b98ee93638c436e2ad` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.947574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc811da9961649b98ee93638c436e2ad)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.954005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.017535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0332dc3e-a2f8-4638-8235-af5b22f7d007 None None] MFA Rules not processed for user `ffbbb7c1637448299beff15d42fa6c81`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:50.018837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0332dc3e-a2f8-4638-8235-af5b22f7d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.077942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc811da9961649b98ee93638c436e2ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:50.081069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.098057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc811da9961649b98ee93638c436e2ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:50.099026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc811da9961649b98ee93638c436e2ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:50.105612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc811da9961649b98ee93638c436e2ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:50.136017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc811da9961649b98ee93638c436e2ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:50.141185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.172366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 881/1797] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:49 2022] DELETE /identity/v3/users/fc811da9961649b98ee93638c436e2ad => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:50.179156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.183726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.197287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 917/1798] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:50.205944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.206344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.206894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.239108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:50.240456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.305183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf788f5-1599-49d9-9078-d9861c7568cd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.315503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.316411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.316864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.317704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.318520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:50.319342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.321284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 918/1799] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:50.323196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.324366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.327725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.340405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c6af5-8c06-4e4c-8507-e7e783dd2f1b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.347172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.347899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.348183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.348575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.349285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.351523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.352771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 882/1800] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:50.359524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.385551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.403967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 919/1801] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/projects => generated 383 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:50.405608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.449933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14561358-f018-4748-ba38-e880d5b8d138 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.531850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.532583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.532855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.533169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.538373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de73f27-39db-4106-92b7-39b377e14484 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.546980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.550487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.551796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.553493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.555367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.556330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.557850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.562246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.575416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 883/1802] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:50.576582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.588856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450c91af-4865-4a43-ab7a-5e32b73c933c service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.596616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.597375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.597712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.598015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.600136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.602698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.623682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 920/1803] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/projects => generated 379 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:50.629753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.642901 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 884/1804] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/users => generated 298 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:50.645115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.655169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.662309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb9eaad-60ca-4b06-9f02-9803746f0807 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.779446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.779946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.780391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.780761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.781632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.785061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.800008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.812361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 885/1805] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:50.818851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.833045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb286d-67f0-4322-b914-6263322a240b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.847348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.848571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.849404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.850011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] PATH_INFO: `/v3/projects/aa52c34b662f40f3a91556a30b702da4/users/8c530f4f1a3643dd8832e2ed79364e63/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.855113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86b155-f5c6-4300-b85c-c53f898e9573 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.873731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.874972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.875543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.876024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.877440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.884120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.908997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa52c34b662f40f3a91556a30b702da4, user_id=8c530f4f1a3643dd8832e2ed79364e63, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.916005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.945643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 921/1806] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/users => generated 290 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:50.951582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.965727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 886/1807] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:50 2022] PUT /identity/v3/projects/aa52c34b662f40f3a91556a30b702da4/users/8c530f4f1a3643dd8832e2ed79364e63/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:50.969038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.976449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f7478-afd3-4fd1-90d0-4182a8cdadda admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.185130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.185130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.185130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.185130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.185843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.191404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.227230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 922/1808] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:51.231977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.274708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4fabb-7585-4580-aade-738c02670def admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.280386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adefcf6-2610-42bd-ac86-75b44082ca67 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.298586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.300781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.302405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.302946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.303899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.307042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.311010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.312240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.312728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.313221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.314432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.317015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.336732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 887/1809] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:51.341043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.344569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:51.351442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 923/1810] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:51.353982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.375178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be49ce-669b-4b4a-8b36-0a1f68930fb1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.385849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.401900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.412748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fba9ad-5a82-403a-8df6-d5062f0e96fe admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.426250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.426664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] PATH_INFO: `/v3/projects/d63fc47cba2e439d96e19dee9f26b90c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.431495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63fc47cba2e439d96e19dee9f26b90c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.439887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.455544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.456259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.456543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.456819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] PATH_INFO: `/v3/projects/aa52c34b662f40f3a91556a30b702da4/users/8c530f4f1a3643dd8832e2ed79364e63/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.459077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa52c34b662f40f3a91556a30b702da4, user_id=8c530f4f1a3643dd8832e2ed79364e63, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.462467 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.492547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 924/1811] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:51 2022] PUT /identity/v3/projects/aa52c34b662f40f3a91556a30b702da4/users/8c530f4f1a3643dd8832e2ed79364e63/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:51.493362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.633038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63fc47cba2e439d96e19dee9f26b90c', 'request_id': 'req-42e22b0d-52ba-4cfb-a2ba-432f8705682a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:51.643240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63fc47cba2e439d96e19dee9f26b90c', 'request_id': 'req-42e22b0d-52ba-4cfb-a2ba-432f8705682a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:51.672572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63fc47cba2e439d96e19dee9f26b90c', 'request_id': 'req-42e22b0d-52ba-4cfb-a2ba-432f8705682a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:51.681250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:51.703186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] The token cache is being invalidate because project d63fc47cba2e439d96e19dee9f26b90c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:51.703533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:51.704378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-42e22b0d-52ba-4cfb-a2ba-432f8705682a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:51.706118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 888/1812] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:51 2022] DELETE /identity/v3/projects/d63fc47cba2e439d96e19dee9f26b90c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:51.711356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.711356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.711356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.738748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845e5b1-d477-40e1-a4f8-22ebc95aa546 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.752938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] MFA Rules not processed for user `8c530f4f1a3643dd8832e2ed79364e63`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:51.754822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:51.970113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 889/1813] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:51.973133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.985371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.089054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.127874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.133500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.133909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.134274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] PATH_INFO: `/v3/projects/13fd4c9bc30342fd9dabdfd7c1f1e09d/users/e507a3b14e1140acafaa60701a62aff4/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.180095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13fd4c9bc30342fd9dabdfd7c1f1e09d, user_id=e507a3b14e1140acafaa60701a62aff4, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.182861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.231663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 925/1814] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:51 2022] PUT /identity/v3/projects/13fd4c9bc30342fd9dabdfd7c1f1e09d/users/e507a3b14e1140acafaa60701a62aff4/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 738 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.232270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.234634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71ac71-9fc1-4bf3-8f1e-3afaf95f3439 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.241412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.324805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.327371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.327910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.328445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] PATH_INFO: `/v3/users/9adc0fae013e45bc8c3554592c14b588` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.331387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e962f-ccfb-4b47-bc18-82f9702d74d9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.331580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adc0fae013e45bc8c3554592c14b588)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.335433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.335907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.337481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.337761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.338029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.376754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.415692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adc0fae013e45bc8c3554592c14b588'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:52.445023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adc0fae013e45bc8c3554592c14b588'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:52.449920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adc0fae013e45bc8c3554592c14b588'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:52.453093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adc0fae013e45bc8c3554592c14b588'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:52.471086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.473288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8624602-842e-4b71-bfed-ebc06e9b04ae service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.516563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 926/1815] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:52.519030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.523006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adc0fae013e45bc8c3554592c14b588'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:52.523281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.525397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.576261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 890/1816] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:51 2022] DELETE /identity/v3/users/9adc0fae013e45bc8c3554592c14b588 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.609519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] MFA Rules not processed for user `e507a3b14e1140acafaa60701a62aff4`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:52.617998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.713913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 927/1817] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:52.719550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.723371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.725380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.726318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.789084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.944380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b8148f-3ec2-4f83-a9dd-d430d376c094 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.963745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.966815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.968184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.969883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.977228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.994579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f0183-476e-452c-b982-a4001e6f726b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.999777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.000573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.001213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.001559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.002540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.005198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.057085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 891/1818] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:52 2022] POST /identity/v3/projects => generated 381 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:53.062497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.234268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.235867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.265965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15cff72-991f-4b0e-9a4e-354fb3433330 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.270011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 928/1819] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:53.275197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.275981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.276148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.276428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.277157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.279501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.325542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 892/1820] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:53 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:53.333023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.343718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad622e9b-f919-44be-abdc-47b099a93d68 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.359677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.360610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.361974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.362397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.364931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.367633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.400179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 929/1821] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:53.404394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.421379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02813b65-8716-4276-ad58-da43e2334af4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.449393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.450627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.452195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.452714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] PATH_INFO: `/v3/projects/49650cc9985e46eda14677c6f9b1e3aa/users/262b64009dca464fa1d28c04d9879da9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.541534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49650cc9985e46eda14677c6f9b1e3aa, user_id=262b64009dca464fa1d28c04d9879da9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.549780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.567411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:53.601420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 893/1822] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:53 2022] PUT /identity/v3/projects/49650cc9985e46eda14677c6f9b1e3aa/users/262b64009dca464fa1d28c04d9879da9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:53.604862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.620789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b519f9-7a2b-4545-a5b7-0659103dd3f0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.762649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.763525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.764376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.764796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.766325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.769679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0069ad-a932-42d5-b9e8-7e2715226b15 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.787885 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 930/1823] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:53.792486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.824472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fb043-9ae9-44e1-9b37-1a1fd80a6971 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.844924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.848174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.849724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.850019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] PATH_INFO: `/v3/projects/49650cc9985e46eda14677c6f9b1e3aa/users/262b64009dca464fa1d28c04d9879da9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.897121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49650cc9985e46eda14677c6f9b1e3aa, user_id=262b64009dca464fa1d28c04d9879da9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.903289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4173041-1d1e-48d4-993c-01ea7b03f5b1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.948923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 894/1824] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:53 2022] PUT /identity/v3/projects/49650cc9985e46eda14677c6f9b1e3aa/users/262b64009dca464fa1d28c04d9879da9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:53.956868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.957497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.957497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.963835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c117ce-eed4-4637-845d-bea2b58fda5f None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.964132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c117ce-eed4-4637-845d-bea2b58fda5f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.964405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c117ce-eed4-4637-845d-bea2b58fda5f None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.966729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 895/1825] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:53.991364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.991646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.992359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:54.092142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:54.097374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.116774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] MFA Rules not processed for user `262b64009dca464fa1d28c04d9879da9`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:54.128037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.346404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:54.372445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 896/1826] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:54.408040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 931/1827] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:54.418482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:54.420608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.528621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa3e64-97d6-477e-8e3a-c3773d2a7c3c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:54.538023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:54.538727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:54.538933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:54.539198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:54.560136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:54.565345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:54.584205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 897/1828] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:54.916686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:54.918162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:54.975695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b8a4b-0d0a-40d1-b087-c73c2cbdf230 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:54.986886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:54.992878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:54.993460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:54.994077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.004433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.007955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.040790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 932/1829] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:55.058173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.059978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.210888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a06fcb-557d-4014-8aa0-96c9208b0692 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.238451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.238451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.238451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.238451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] PATH_INFO: `/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.252159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e44ef1642a204d1c9cc3a849d84c7c87)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.269140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.381367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:55.468436 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:55.512939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e44ef1642a204d1c9cc3a849d84c7c87', 'request_id': 'req-44b90f46-ba48-4346-8677-f3cad29757f0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.517541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ba3370-0059-4e98-91f1-c4d5616bfa5a service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:55.546088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e44ef1642a204d1c9cc3a849d84c7c87', 'request_id': 'req-44b90f46-ba48-4346-8677-f3cad29757f0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.578946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e44ef1642a204d1c9cc3a849d84c7c87', 'request_id': 'req-44b90f46-ba48-4346-8677-f3cad29757f0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.626502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] The token cache is being invalidate because project e44ef1642a204d1c9cc3a849d84c7c87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:55.627502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.629436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44b90f46-ba48-4346-8677-f3cad29757f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.637839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 898/1830] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:55 2022] DELETE /identity/v3/projects/e44ef1642a204d1c9cc3a849d84c7c87 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:55.645514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d251-8da0-44e1-926c-952a5c8280e6 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.648683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d251-8da0-44e1-926c-952a5c8280e6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.649489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5777d251-8da0-44e1-926c-952a5c8280e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.736622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5777d251-8da0-44e1-926c-952a5c8280e6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.739408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5777d251-8da0-44e1-926c-952a5c8280e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:55.785742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:55.788042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.788504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.789115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] PATH_INFO: `/v3/users/e507a3b14e1140acafaa60701a62aff4` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.792082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e507a3b14e1140acafaa60701a62aff4)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:55.797130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:55.937381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 899/1831] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:55.941706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e507a3b14e1140acafaa60701a62aff4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.946664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:55.946986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:55.947296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:55.968314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e507a3b14e1140acafaa60701a62aff4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.972962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e507a3b14e1140acafaa60701a62aff4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.973942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e507a3b14e1140acafaa60701a62aff4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:55.993966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:55.997842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4750fb6d-0ea8-49ff-b025-052124e81074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e507a3b14e1140acafaa60701a62aff4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.003913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.028377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 900/1832] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:56.036288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.055947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46b512-e00f-4ecc-89a0-a4270e1075ee None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.062995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 933/1833] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:55 2022] DELETE /identity/v3/users/e507a3b14e1140acafaa60701a62aff4 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:56.065006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.065495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.065928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.071729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.073404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.073827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.074329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.075655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:56.076139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.081991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.106811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:56.112969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.120953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 901/1834] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:56.127980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.198556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413442a4-4511-46ab-a250-f7dc5d86711f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.287814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:56.319040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 934/1835] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:56.383159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.383671 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.384934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.385726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.385893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.386405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.386703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.390124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.474786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 902/1836] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:56 2022] POST /identity/v3/projects => generated 365 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:56.477219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.506842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:56.544440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c98a0-9db2-4184-b3b7-6c404c06c41e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.595896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c2cd56-f818-4208-975b-d07dd233ff7e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.604749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.605434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.605715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.606377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.607035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.612003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.641294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.642027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.642272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.642585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.647406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 903/1837] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:56 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:56.661996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.666215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.670132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.683957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9057baf6-217b-4919-a97c-480cceffc872 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.701985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.702700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.703001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.703269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.703931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.706254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.717841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 935/1838] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:56.729791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 904/1839] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:56.735193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.754568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3742c4-76fc-4001-b5e8-c5310a8249d0 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.780630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.785640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.786545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.787314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.789024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.795376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.840801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 936/1840] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:56.841189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.868320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5470fdc-496d-4e21-aa3f-133867fb3dcd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.879786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.887043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:56.899184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.902220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.902665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.903005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] PATH_INFO: `/v3/projects/685a92bc38e14a5592f9be2f4956aacf/users/37691ccc7b8c4edea86912d5a08f65dc/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.932459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2642bdc-351d-464b-b89d-dd60785afe7e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.936392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=685a92bc38e14a5592f9be2f4956aacf, user_id=37691ccc7b8c4edea86912d5a08f65dc, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.944715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.946057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.946670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.947408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] PATH_INFO: `/v3/users/ffbbb7c1637448299beff15d42fa6c81` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.949822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbbb7c1637448299beff15d42fa6c81)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.950189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dce0c-05f6-4604-92d1-6d0e636891ca admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.954455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.013759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 905/1841] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:56 2022] PUT /identity/v3/projects/685a92bc38e14a5592f9be2f4956aacf/users/37691ccc7b8c4edea86912d5a08f65dc/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:57.017386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.018184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.018800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.084111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] MFA Rules not processed for user `37691ccc7b8c4edea86912d5a08f65dc`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:57.087767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.095180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbbb7c1637448299beff15d42fa6c81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.116513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbbb7c1637448299beff15d42fa6c81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.118158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbbb7c1637448299beff15d42fa6c81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.120122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbbb7c1637448299beff15d42fa6c81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.136073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2363b9e7-40c3-48bc-a721-6c74ebe77200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbbb7c1637448299beff15d42fa6c81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.177491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 937/1842] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:56 2022] DELETE /identity/v3/users/ffbbb7c1637448299beff15d42fa6c81 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:57.185203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.185688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.186253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.261166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:57.267972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.285890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.329534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 906/1843] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:57.332328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.334637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.472656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 938/1844] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:57.509751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.521221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da5110-882c-4ac6-a97b-83af9570e43e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.535591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.536663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.537134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.538177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.539391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.543120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.594566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.599566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.617046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 907/1845] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/projects => generated 363 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:57.618627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.776805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5393f-8183-4ff4-ae3b-b4068dfba1de None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.791980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d26b1-542f-436e-bb5b-1effa74e9a53 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.803516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.804436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.804778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.806042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.806730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.809826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.861872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 908/1846] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:57.865233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.878806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e168791-da21-4526-bc63-673bc536b139 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.889305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.890178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.890737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.891187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.892262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.896740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.904805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.906323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.906758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.907289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.927033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.929241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 909/1847] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:57.929887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.933304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.954099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 939/1848] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:57.957237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973945cc-97e5-4d52-bc2c-aebbbf246ebb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.964006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.964985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.965401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.965920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] PATH_INFO: `/v3/projects/6d0bf70dd74f474480053f52c0104d54/users/3ef0b54a163f49f9a49dc05a2f252064/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.994467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0bf70dd74f474480053f52c0104d54, user_id=3ef0b54a163f49f9a49dc05a2f252064, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.999690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.031000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.032979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 910/1849] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:57 2022] PUT /identity/v3/projects/6d0bf70dd74f474480053f52c0104d54/users/3ef0b54a163f49f9a49dc05a2f252064/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:58.053070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6ecbe8-9a43-4bc5-829e-1eb772d8b15f service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.182483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.183733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.184443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.184983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.186795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.191108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.216768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 940/1850] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:58.218676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.230040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.234876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.238556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a17eb-ee26-4fad-b3d2-e4c85e50fe70 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.258956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.263861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.265320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.265986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.266611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] PATH_INFO: `/v3/projects/6d0bf70dd74f474480053f52c0104d54/users/3ef0b54a163f49f9a49dc05a2f252064/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.267143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c569264-611d-4fc1-8761-5a91cb07ad23 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.271362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0bf70dd74f474480053f52c0104d54, user_id=3ef0b54a163f49f9a49dc05a2f252064, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.278933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe39bf9-f6be-4e60-96d2-9b2d8c533f2a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.291940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.295034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.297719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.298484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] PATH_INFO: `/v3/projects/13fd4c9bc30342fd9dabdfd7c1f1e09d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.306505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] RBAC: Authorizing `identity:delete_project(project_id=13fd4c9bc30342fd9dabdfd7c1f1e09d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.312166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.313165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 911/1851] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:58 2022] PUT /identity/v3/projects/6d0bf70dd74f474480053f52c0104d54/users/3ef0b54a163f49f9a49dc05a2f252064/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.333549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.333933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.334511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.387050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] MFA Rules not processed for user `3ef0b54a163f49f9a49dc05a2f252064`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:58.388443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.398015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13fd4c9bc30342fd9dabdfd7c1f1e09d', 'request_id': 'req-10615f11-1a96-4ee9-b2e9-e065d2da92db'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.408127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13fd4c9bc30342fd9dabdfd7c1f1e09d', 'request_id': 'req-10615f11-1a96-4ee9-b2e9-e065d2da92db'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.443281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13fd4c9bc30342fd9dabdfd7c1f1e09d', 'request_id': 'req-10615f11-1a96-4ee9-b2e9-e065d2da92db'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.463137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] The token cache is being invalidate because project 13fd4c9bc30342fd9dabdfd7c1f1e09d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:58.463438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.466967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.474095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 941/1852] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/projects/13fd4c9bc30342fd9dabdfd7c1f1e09d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.475361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.478524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.531085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 912/1853] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:58.546089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.547813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.562930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.617004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10615f11-1a96-4ee9-b2e9-e065d2da92db admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.625790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.626793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.627256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.627579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] PATH_INFO: `/v3/users/262b64009dca464fa1d28c04d9879da9` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.629329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=262b64009dca464fa1d28c04d9879da9)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.632353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.636772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dfe11-bd2e-4ba1-b6c2-101fa987c2cb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.640981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.641861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.642143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.642413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.653946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.655477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.668558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 913/1854] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:58.669450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.671150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.689066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b37e5-2f0f-4eb4-8a3f-d086a3a95e4b service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.695414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.696229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.696514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.696786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] PATH_INFO: `/v3/projects/133c15d5ab28474dbacd9935d10821f7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.698041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=133c15d5ab28474dbacd9935d10821f7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.700280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.701747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '262b64009dca464fa1d28c04d9879da9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.713668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '262b64009dca464fa1d28c04d9879da9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.714492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '262b64009dca464fa1d28c04d9879da9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.715389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '262b64009dca464fa1d28c04d9879da9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.728814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '262b64009dca464fa1d28c04d9879da9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.790868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 942/1855] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/users/262b64009dca464fa1d28c04d9879da9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.803038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '133c15d5ab28474dbacd9935d10821f7', 'request_id': 'req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.811690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '133c15d5ab28474dbacd9935d10821f7', 'request_id': 'req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.840742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '133c15d5ab28474dbacd9935d10821f7', 'request_id': 'req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.881801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] The token cache is being invalidate because project 133c15d5ab28474dbacd9935d10821f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:58.882276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.883804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.887668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 914/1856] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/projects/133c15d5ab28474dbacd9935d10821f7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.892141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.894289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.910298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.060484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681bce9b-90ab-4f35-8ab8-e7090c5f66bb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.070737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.071746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.072414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.073319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] PATH_INFO: `/v3/users/8c530f4f1a3643dd8832e2ed79364e63` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.076263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c530f4f1a3643dd8832e2ed79364e63)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.080148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.250867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c530f4f1a3643dd8832e2ed79364e63'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.273378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c530f4f1a3643dd8832e2ed79364e63'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.273378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c530f4f1a3643dd8832e2ed79364e63'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.273378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c530f4f1a3643dd8832e2ed79364e63'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.279886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.284718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c530f4f1a3643dd8832e2ed79364e63'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.285510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.350214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 943/1857] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/users/8c530f4f1a3643dd8832e2ed79364e63 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.511551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5571b-7b82-4cf3-b8fa-06ff76186c84 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.533022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.534544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.587994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.638784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.641475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.642731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.645028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] PATH_INFO: `/v3/projects/49650cc9985e46eda14677c6f9b1e3aa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.649286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49650cc9985e46eda14677c6f9b1e3aa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.656043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.659831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7527443-d850-464c-add7-4e849227d925 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.666095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.668198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.668198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.668509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] PATH_INFO: `/v3/users/37691ccc7b8c4edea86912d5a08f65dc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.670120 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37691ccc7b8c4edea86912d5a08f65dc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.672102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.719761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.752497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37691ccc7b8c4edea86912d5a08f65dc'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.766969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37691ccc7b8c4edea86912d5a08f65dc'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.767943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37691ccc7b8c4edea86912d5a08f65dc'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.768656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37691ccc7b8c4edea86912d5a08f65dc'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.779211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f9551d02-b48e-425f-8f79-608fd8a3dc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37691ccc7b8c4edea86912d5a08f65dc'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.816056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 944/1858] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:59 2022] DELETE /identity/v3/users/37691ccc7b8c4edea86912d5a08f65dc => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:59.825048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.825546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.827006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.830328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49650cc9985e46eda14677c6f9b1e3aa', 'request_id': 'req-fd77c64d-aa9b-4173-b814-ddd1cb256197'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.844596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49650cc9985e46eda14677c6f9b1e3aa', 'request_id': 'req-fd77c64d-aa9b-4173-b814-ddd1cb256197'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.860227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49650cc9985e46eda14677c6f9b1e3aa', 'request_id': 'req-fd77c64d-aa9b-4173-b814-ddd1cb256197'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.873259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.877555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.892059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] The token cache is being invalidate because project 49650cc9985e46eda14677c6f9b1e3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:59.892712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.894411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fd77c64d-aa9b-4173-b814-ddd1cb256197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.899045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 915/1859] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:59 2022] DELETE /identity/v3/projects/49650cc9985e46eda14677c6f9b1e3aa => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.909283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.909834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.910462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.963885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.965349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.025839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 945/1860] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.025839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.029295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.046712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751c628-7fdf-477a-b56e-91c26e7f5858 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.052020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.052664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.052974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.053254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] PATH_INFO: `/v3/projects/aa52c34b662f40f3a91556a30b702da4` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.054384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa52c34b662f40f3a91556a30b702da4)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.056688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.063435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 916/1861] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.065094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.068779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.156718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa52c34b662f40f3a91556a30b702da4', 'request_id': 'req-9e56be6f-0f56-49f1-a755-ef65985d8c49'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.183030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa52c34b662f40f3a91556a30b702da4', 'request_id': 'req-9e56be6f-0f56-49f1-a755-ef65985d8c49'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.196279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18224b29-2d96-44c5-88f2-d7027915d356 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.211005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa52c34b662f40f3a91556a30b702da4', 'request_id': 'req-9e56be6f-0f56-49f1-a755-ef65985d8c49'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.243015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.245592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] The token cache is being invalidate because project aa52c34b662f40f3a91556a30b702da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:00.245987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.246922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9e56be6f-0f56-49f1-a755-ef65985d8c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.253529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 946/1862] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:00 2022] DELETE /identity/v3/projects/aa52c34b662f40f3a91556a30b702da4 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:00.253529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e7da-ac5a-4a1d-97c2-845ac68a1ec0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.253529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e7da-ac5a-4a1d-97c2-845ac68a1ec0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.253529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8e7da-ac5a-4a1d-97c2-845ac68a1ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.287875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-09e8e7da-ac5a-4a1d-97c2-845ac68a1ec0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.290233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-09e8e7da-ac5a-4a1d-97c2-845ac68a1ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.294035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.295328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.295819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.296299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.302048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.377039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.445688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 947/1863] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.447722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd254-834a-4abc-8f17-fee51e086953 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.448188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd254-834a-4abc-8f17-fee51e086953 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.448492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88cd254-834a-4abc-8f17-fee51e086953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.460307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.464383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.480170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a88cd254-834a-4abc-8f17-fee51e086953 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.482747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a88cd254-834a-4abc-8f17-fee51e086953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.491415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 917/1864] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:00.493484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.495372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 948/1865] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.505192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.505985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.506365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.516364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2437-0198-4044-a63f-6846fb8b2f8a service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.525648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.526894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.527382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.527915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.530256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:00.530935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.531188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.532369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.534500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.543298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 949/1866] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.547637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.561478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa018d-53fc-4a6a-a371-e561f84c3024 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.572725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 918/1867] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:00.573395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.574757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.575436 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.576419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.577900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.578452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:00.579633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.583326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.595079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff27a7d-7192-4369-bb29-2ef433172859 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.601595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.602506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.603019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.603410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.604148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.607239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.618592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 950/1868] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:00.624438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.656401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80b9b5-905c-48c3-9dbf-ca3f8a6db8d7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.671247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.671929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 919/1869] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:00.673605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.674454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.674945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.676157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.676924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.679352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa277de-6b3c-4649-b12c-798a28559425 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.774901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 951/1870] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:00.777434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6c7b-e75f-4f6d-ae2b-8760fb853a92 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.778397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6c7b-e75f-4f6d-ae2b-8760fb853a92 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.780196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e6c7b-e75f-4f6d-ae2b-8760fb853a92 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.783456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 952/1871] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:51:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:00.785396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e6c7b-e75f-4f6d-ae2b-8760fb853a92 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.863398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2f0e7-097f-4dc4-8355-e955eb7e3541 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.003505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.049104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.051676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.052141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.053484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.055714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.062517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4278-792b-4582-8e3e-b090ff982670 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.100484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e6c7b-e75f-4f6d-ae2b-8760fb853a92 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.120229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.120985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.121727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.122179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.122863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.124687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 920/1872] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/users => generated 293 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:01.126308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.126679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.127017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.131119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.171339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.181273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 953/1873] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/users => generated 294 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:01.181273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.182340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.200608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515fc22-cfd2-430d-b467-4c1458ef9a46 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.212784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.226315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.226707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.227064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.228675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.230639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.252685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 954/1874] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:01.253951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.278468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d5619-3183-422b-9c72-e0c1c1520f64 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.313325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.315838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.316422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.316683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.318749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.322241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.339378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 955/1875] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:01.340920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.343192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 921/1876] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.344092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.368060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1f67-0952-4d5e-8b2a-8b79241f862b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.368558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be2f57-f262-4fda-a501-6c0b26e353e1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.382954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.383606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.383959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.384284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.384987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.387439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.388452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.389070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.389635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.390074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.391112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.393656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.418465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 922/1877] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:01.420150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.434045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 956/1878] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:01.439942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.443083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f03aa-b134-40cc-8acd-161c2712d79b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.459571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.460268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.460551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.460826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] PATH_INFO: `/v3/projects/5c4008693daa4cc596b4c1d71a288a1e/users/19d3934a09e7419191c185c2617d0f1c/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.471801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5adafd5-daa6-43b6-9ef5-0987563d9754 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.484235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.484983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.485273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.485549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] PATH_INFO: `/v3/projects/31f84e38981c47d0aa4d52688fc0d45f/users/de2475714b6f4d15b132e7d16a889344/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.497951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c4008693daa4cc596b4c1d71a288a1e, user_id=19d3934a09e7419191c185c2617d0f1c, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.501321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.525705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f84e38981c47d0aa4d52688fc0d45f, user_id=de2475714b6f4d15b132e7d16a889344, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.534448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9fa21-0eee-430d-ba6a-134b48223b98 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.557023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 923/1879] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:01 2022] PUT /identity/v3/projects/5c4008693daa4cc596b4c1d71a288a1e/users/19d3934a09e7419191c185c2617d0f1c/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:01.557665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 957/1880] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:01 2022] PUT /identity/v3/projects/31f84e38981c47d0aa4d52688fc0d45f/users/de2475714b6f4d15b132e7d16a889344/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:01.559582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.562318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.573007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0481c-7686-457d-b001-e92e65b790f7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.573681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0481c-7686-457d-b001-e92e65b790f7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.574450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0481c-7686-457d-b001-e92e65b790f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.643811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.650431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-38f0481c-7686-457d-b001-e92e65b790f7 None None] MFA Rules not processed for user `19d3934a09e7419191c185c2617d0f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.654010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-38f0481c-7686-457d-b001-e92e65b790f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.701810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5f738-5ed3-440b-bb34-137027be84ae admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.751172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.819343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 958/1881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:01.821706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.822263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.822594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.823921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.824698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.825154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.825509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.836960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.838776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.851945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 924/1882] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:01.853360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.855468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.862252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] MFA Rules not processed for user `de2475714b6f4d15b132e7d16a889344`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.863893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.974722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e6564-7872-4536-9f86-00f6cf3192a6 service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.977917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 959/1883] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.978796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.992847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.993673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.994051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.994272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] PATH_INFO: `/v3/projects/685a92bc38e14a5592f9be2f4956aacf` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.995509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=685a92bc38e14a5592f9be2f4956aacf)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.997908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56571860-592b-44a1-acd9-a0df57164ad3 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.999609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.003468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.003739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.003904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.004171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.004806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.006859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.074647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 960/1884] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:02.075688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.077853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.097388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '685a92bc38e14a5592f9be2f4956aacf', 'request_id': 'req-e9597c3d-c178-428e-a99b-c763b64b71e9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.135794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '685a92bc38e14a5592f9be2f4956aacf', 'request_id': 'req-e9597c3d-c178-428e-a99b-c763b64b71e9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.153430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '685a92bc38e14a5592f9be2f4956aacf', 'request_id': 'req-e9597c3d-c178-428e-a99b-c763b64b71e9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.172998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] The token cache is being invalidate because project 685a92bc38e14a5592f9be2f4956aacf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:02.173286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.174699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.178328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 925/1885] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:01 2022] DELETE /identity/v3/projects/685a92bc38e14a5592f9be2f4956aacf => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:02.179200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.182059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.183546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed732e-7657-4416-a221-e8edd89c95aa admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.191753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.193531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.193835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.194112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.198283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.247569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.265030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.282717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9597c3d-c178-428e-a99b-c763b64b71e9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.292420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.294122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.306419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 961/1886] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:02.307115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.308744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.375942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.375942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.376610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.376610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.377157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.379174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.390448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c8e95-1297-4757-95b0-b9e282164867 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.396591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.397348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.397563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.397958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] PATH_INFO: `/v3/users/3ef0b54a163f49f9a49dc05a2f252064` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.399105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef0b54a163f49f9a49dc05a2f252064)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.401082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.408523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 926/1887] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/users => generated 301 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:02.410171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.413081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.434215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0e287-8a7b-4e34-86f3-04a174537d12 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.442826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.443977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.444646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.445473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] PATH_INFO: `/v3/users/de2475714b6f4d15b132e7d16a889344` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.447731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de2475714b6f4d15b132e7d16a889344)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.451628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.467904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef0b54a163f49f9a49dc05a2f252064'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.482181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef0b54a163f49f9a49dc05a2f252064'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.482900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef0b54a163f49f9a49dc05a2f252064'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.483667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef0b54a163f49f9a49dc05a2f252064'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.511771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef0b54a163f49f9a49dc05a2f252064'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.528811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de2475714b6f4d15b132e7d16a889344'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.532051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 962/1888] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:02 2022] DELETE /identity/v3/users/3ef0b54a163f49f9a49dc05a2f252064 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:02.533142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.535073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.548055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de2475714b6f4d15b132e7d16a889344'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.549159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de2475714b6f4d15b132e7d16a889344'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.550164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de2475714b6f4d15b132e7d16a889344'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.571193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1d5d4dd4-eec5-49d1-b2b9-dca6f69c020d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de2475714b6f4d15b132e7d16a889344'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.587335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.595106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 927/1889] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:02 2022] DELETE /identity/v3/users/de2475714b6f4d15b132e7d16a889344 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:02.602255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.603471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.603933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.611675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b24a76-671a-493f-96a2-12d21cd569d6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.635877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:02.637126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.680522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.681385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.681680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.681928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.683045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.685546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.700967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 963/1890] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:02.705337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.707391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.724993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7070c-c6fc-4960-a720-35084c877df9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.731073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.732232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.732644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.732903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.733346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] PATH_INFO: `/v3/projects/83032447af984380a4fd6bffebe852bc/users/50669ea506ed422c8e3fa1f23e78ffb4/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.748225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 928/1891] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:02.757883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83032447af984380a4fd6bffebe852bc, user_id=50669ea506ed422c8e3fa1f23e78ffb4, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.760536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.770165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.774090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.781915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 964/1892] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:02 2022] PUT /identity/v3/projects/83032447af984380a4fd6bffebe852bc/users/50669ea506ed422c8e3fa1f23e78ffb4/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:02.786891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.899878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11e6506-6bed-4ed5-8f55-1633ed14c752 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.923455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e2e46-dcfd-47d8-bfd5-10ffd2786e35 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.933769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.935225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.935839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.936684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.938068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.941760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.974339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 965/1893] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:02.977603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.003478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d9aaa-b339-4270-9350-181d2cfd2cbb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.009897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.010769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.011140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.011566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] PATH_INFO: `/v3/projects/83032447af984380a4fd6bffebe852bc/users/50669ea506ed422c8e3fa1f23e78ffb4/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.014664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83032447af984380a4fd6bffebe852bc, user_id=50669ea506ed422c8e3fa1f23e78ffb4, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.019048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.020085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fe743-e077-41c5-b8f4-6792303f867a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.020208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.020710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.021577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.052539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.057178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.081209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 966/1894] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:02 2022] PUT /identity/v3/projects/83032447af984380a4fd6bffebe852bc/users/50669ea506ed422c8e3fa1f23e78ffb4/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:03.085409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.085695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.085964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.092270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 929/1895] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:03.151237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] MFA Rules not processed for user `50669ea506ed422c8e3fa1f23e78ffb4`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:03.152622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.230288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:03.313780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 967/1896] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:03.384859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.388408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.493811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.495376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.585332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c5abcd-4762-4d70-8c21-6df95e07d841 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.684761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165dc6f9-2dc9-4e21-8242-f3464bbf903f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.698028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.699082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.699545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.699977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] PATH_INFO: `/v3/projects/6d0bf70dd74f474480053f52c0104d54` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.702851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0bf70dd74f474480053f52c0104d54)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.706352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.824408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.825590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.825873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.826141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.841177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0bf70dd74f474480053f52c0104d54', 'request_id': 'req-a5f74b89-a000-460b-aaef-f367d3b6a18f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.842321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.846114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cdb07f-b85b-49e1-bf45-fcfbecad43e1 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.859744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0bf70dd74f474480053f52c0104d54', 'request_id': 'req-a5f74b89-a000-460b-aaef-f367d3b6a18f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.873809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 930/1897] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:03.886225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0bf70dd74f474480053f52c0104d54', 'request_id': 'req-a5f74b89-a000-460b-aaef-f367d3b6a18f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.909368 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] The token cache is being invalidate because project 6d0bf70dd74f474480053f52c0104d54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:03.909705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.913917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a5f74b89-a000-460b-aaef-f367d3b6a18f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.918388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 968/1898] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:03 2022] DELETE /identity/v3/projects/6d0bf70dd74f474480053f52c0104d54 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:03.923909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549244-ec77-4644-876d-f8633da04a2a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.924207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549244-ec77-4644-876d-f8633da04a2a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.924461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47549244-ec77-4644-876d-f8633da04a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.963204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-47549244-ec77-4644-876d-f8633da04a2a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:03.964750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-47549244-ec77-4644-876d-f8633da04a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.003102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:04.100948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 931/1899] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:04.106887 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.107526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.107700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.149434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:04.150696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.173605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 969/1900] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:04.178957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549244-ec77-4644-876d-f8633da04a2a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.217240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47549244-ec77-4644-876d-f8633da04a2a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.227011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.227958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.228638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.229119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.230196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:04.230676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.233694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.264521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.270022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.271292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 932/1901] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:04.272482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.284545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7e4a5-266f-4e74-abcc-e00af39f5a41 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.287729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfb8e16-15f7-4f60-9491-9244ad21960e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.291616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.292943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.293208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.293480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.294148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.296527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.298963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.300034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.300538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.301062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] PATH_INFO: `/v3/users/19d3934a09e7419191c185c2617d0f1c` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.306409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d3934a09e7419191c185c2617d0f1c)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.311295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.337056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 933/1902] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:04.340921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.412528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d3934a09e7419191c185c2617d0f1c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:04.421901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:04.440969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d3934a09e7419191c185c2617d0f1c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:04.443371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d3934a09e7419191c185c2617d0f1c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:04.444998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d3934a09e7419191c185c2617d0f1c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:04.458322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8566a1df-fa04-40af-9b83-6a2dc082ae3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d3934a09e7419191c185c2617d0f1c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:04.471663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab5ed-75f3-406f-a73c-7eebb455bf15 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.483144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.484935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.484935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.485249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.485701 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.490935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.503771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 970/1903] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:04 2022] DELETE /identity/v3/users/19d3934a09e7419191c185c2617d0f1c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:04.511785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.512178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.512622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.536074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:04.537445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.538916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 934/1904] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:04.542164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.544784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.559941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca8bf9-4229-4f35-8e8e-865fa2c52e4f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.689014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 971/1905] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:04.689292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.703066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.703429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.704228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.704647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.704808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.708514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.715782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.762692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 935/1906] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:04.770910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.810420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffef47c-5bbb-4487-af59-63da2b34d639 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.842205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737cd1b-b8bf-442d-b1de-466d9ac42a1e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.852656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.853676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.854157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.854767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.859021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.860006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.860497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.861093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.862278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.865479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f1995-7b68-4ca3-927e-97a66a7ee2e7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.870067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:04.873151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.875649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.893355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 972/1907] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:04.904242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.908580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 936/1908] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:04.925363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049404f4-dd00-42cd-bfb3-b3db1a483586 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.936602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.938298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.939549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.940327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] PATH_INFO: `/v3/projects/a6ad95d6dd384b82becda0cae061ec7d/users/114aa2e38c5d4580890316f28ce308a4/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.995040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ad95d6dd384b82becda0cae061ec7d, user_id=114aa2e38c5d4580890316f28ce308a4, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.011756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.054284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 973/1909] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:04 2022] PUT /identity/v3/projects/a6ad95d6dd384b82becda0cae061ec7d/users/114aa2e38c5d4580890316f28ce308a4/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:05.066381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.066759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.067103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.106449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] MFA Rules not processed for user `114aa2e38c5d4580890316f28ce308a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:05.108776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.238516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.242097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 937/1910] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:05.248282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.255627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.322152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.359074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d405406-f745-4ceb-b608-89ee7efd4907 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.367376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.368577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.369307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.370113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.370964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.373966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.381979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711bd368-72f5-4a56-85b4-c9eebdd40b75 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.387509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.388748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.389097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.389367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] PATH_INFO: `/v3/projects/31f84e38981c47d0aa4d52688fc0d45f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.390898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f84e38981c47d0aa4d52688fc0d45f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.393244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.415223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 974/1911] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/projects => generated 369 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:05.416408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.468747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f84e38981c47d0aa4d52688fc0d45f', 'request_id': 'req-6efe5755-5184-424b-b84c-5449d7cc214d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.477158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f84e38981c47d0aa4d52688fc0d45f', 'request_id': 'req-6efe5755-5184-424b-b84c-5449d7cc214d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.495749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f84e38981c47d0aa4d52688fc0d45f', 'request_id': 'req-6efe5755-5184-424b-b84c-5449d7cc214d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.531340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] The token cache is being invalidate because project 31f84e38981c47d0aa4d52688fc0d45f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:05.532528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.534007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.537580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 938/1912] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:05 2022] DELETE /identity/v3/projects/31f84e38981c47d0aa4d52688fc0d45f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:05.540033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.544484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.554011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46049da-ffc3-4643-92b9-e0f3b49107c6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.595523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.636989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.637943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.638359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.638844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.645677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.648303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61d006-357c-4b15-99de-3af964dd4820 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.686506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 975/1913] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/users => generated 291 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:05.687884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.688237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.688516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.716628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe5755-5184-424b-b84c-5449d7cc214d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.717276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:05.718721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.725883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.726157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.726483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.726965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] PATH_INFO: `/v3/projects/5c4008693daa4cc596b4c1d71a288a1e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.727922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 976/1914] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:05.728659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.730069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.734636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c4008693daa4cc596b4c1d71a288a1e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.740105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.742962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788e560-badf-44f7-9e2d-a56e4b136c2b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.748753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.749472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.749755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.750035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] PATH_INFO: `/v3/users/0e901d4e55bd4b36b9c45ef4369613e7` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.751236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e901d4e55bd4b36b9c45ef4369613e7)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.753346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.822460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e901d4e55bd4b36b9c45ef4369613e7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.840807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c4008693daa4cc596b4c1d71a288a1e', 'request_id': 'req-de68679d-b265-45ee-ab9a-2a38d86e2fe1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.841073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e901d4e55bd4b36b9c45ef4369613e7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.841773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e901d4e55bd4b36b9c45ef4369613e7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.842452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e901d4e55bd4b36b9c45ef4369613e7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.848228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.853600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c4008693daa4cc596b4c1d71a288a1e', 'request_id': 'req-de68679d-b265-45ee-ab9a-2a38d86e2fe1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.867440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c4008693daa4cc596b4c1d71a288a1e', 'request_id': 'req-de68679d-b265-45ee-ab9a-2a38d86e2fe1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.872380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f6c07329-6a33-4ce9-9d2f-4dec6374bff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e901d4e55bd4b36b9c45ef4369613e7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.891101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] The token cache is being invalidate because project 5c4008693daa4cc596b4c1d71a288a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:05.891645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.893137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:05.900702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 939/1915] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:05 2022] DELETE /identity/v3/projects/5c4008693daa4cc596b4c1d71a288a1e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:05.901824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 977/1916] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:05 2022] DELETE /identity/v3/users/0e901d4e55bd4b36b9c45ef4369613e7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:05.901938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.903692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.903998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.904385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.908755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.947998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:05.950086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.020598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68679d-b265-45ee-ab9a-2a38d86e2fe1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.025742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 978/1917] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:06.026397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.027776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.032731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.033966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.034516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.035062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.036305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.041799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c087f1b-2bbe-4b8e-adc2-ee21224cea40 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.054189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 940/1918] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:06.055874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.056355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.056628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.058038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b2cbd-478e-4308-96b0-6e199ad3d547 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.063087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.063950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.064201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.064440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] PATH_INFO: `/v3/users/50669ea506ed422c8e3fa1f23e78ffb4` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.067080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50669ea506ed422c8e3fa1f23e78ffb4)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.069370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.079876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:06.081137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.092039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 941/1919] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:06.095883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.110535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc23f1b-a314-4c35-80b6-60f126de88d0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.118946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.119879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.120353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.121805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.121805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:06.122117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.123865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.139314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 942/1920] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:06.140176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.152587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeaabca-f3a0-4651-9fd3-c02dfa284edd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.157158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50669ea506ed422c8e3fa1f23e78ffb4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:06.157707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.158469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.158748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.159021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] PATH_INFO: `/v3/projects/fec8b3fda13e473bb07a96d02417bd38/users/b1ad77c9274d41da932c486c223295ff/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.168857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50669ea506ed422c8e3fa1f23e78ffb4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:06.171005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50669ea506ed422c8e3fa1f23e78ffb4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:06.171986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50669ea506ed422c8e3fa1f23e78ffb4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:06.182700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec8b3fda13e473bb07a96d02417bd38, user_id=b1ad77c9274d41da932c486c223295ff, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.183782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50669ea506ed422c8e3fa1f23e78ffb4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:06.185845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.202151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.215505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 943/1921] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:06 2022] PUT /identity/v3/projects/fec8b3fda13e473bb07a96d02417bd38/users/b1ad77c9274d41da932c486c223295ff/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:06.216843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.219633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 979/1922] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:06 2022] DELETE /identity/v3/users/50669ea506ed422c8e3fa1f23e78ffb4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:06.221184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.224665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.227568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.311059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072822f-7188-488e-b2b8-36cc81e15384 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.321789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.350869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd32ba6-3bc5-4d96-a452-3bac546cf2b8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.358781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.359710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.360114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.360516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.361610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.364829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.379505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.380832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.381191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.381427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.383983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.399620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.401325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.412762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 980/1923] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/projects => generated 375 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:06.412762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.413939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.419595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 944/1924] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:06.420730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.435643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf42c8a-e5d0-4401-84aa-13b2cfe33f57 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.524989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2742bbc-e5db-41f9-a5cc-e024ba2dd226 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.529694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.530502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.530783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.531066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.531692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.532509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.533556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.533771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.534003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.534703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.535844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.538984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.552825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 981/1925] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:06.556965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.563846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 945/1926] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:06.568615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.574264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c0d2-80c3-43c8-97bd-e0be787038b5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.579868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.580918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.581032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.581387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] PATH_INFO: `/v3/projects/fec8b3fda13e473bb07a96d02417bd38/users/b1ad77c9274d41da932c486c223295ff/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.581538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e133b-eb82-44be-999a-a776461fa91d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.583691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec8b3fda13e473bb07a96d02417bd38, user_id=b1ad77c9274d41da932c486c223295ff, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.586675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b06fd-2513-4b7a-acc7-ba76223fbdbd admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.587726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.588293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.588664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.589601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.590758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.592874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.604966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 982/1927] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:06 2022] PUT /identity/v3/projects/fec8b3fda13e473bb07a96d02417bd38/users/b1ad77c9274d41da932c486c223295ff/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:06.612356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.612749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.613058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.615577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 946/1928] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:06.620251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.632545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62221bcc-ee08-442f-8877-33f380bff3f9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.650069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] MFA Rules not processed for user `b1ad77c9274d41da932c486c223295ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:06.651409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.683377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.716150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.717152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.717477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.717663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.718615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.720341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.733943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 947/1929] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:06.737838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.744792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 983/1930] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:06.753097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.754905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c10-0f5e-4746-b088-31f7ed0566cb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.757343 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.759832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.760594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.761146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.761467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] PATH_INFO: `/v3/projects/aec86426074249009e3f7f8c246f3744/users/642ab4f52a3b47fe8ffa46170bb393e3/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.781001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec86426074249009e3f7f8c246f3744, user_id=642ab4f52a3b47fe8ffa46170bb393e3, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.788021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb9554-2027-4f50-bb3c-e673a8c92eb7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.816075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 948/1931] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:06 2022] PUT /identity/v3/projects/aec86426074249009e3f7f8c246f3744/users/642ab4f52a3b47fe8ffa46170bb393e3/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:06.830361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.830361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.830361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.832596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.874271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf41ffea-7a59-44eb-a132-d85ec55061d9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.942608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] MFA Rules not processed for user `642ab4f52a3b47fe8ffa46170bb393e3`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:06.944926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.028947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.030014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.030014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.030273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.122943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 949/1932] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:07.124966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.136819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.172935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.175749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.196138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 984/1933] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:07.199015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.201524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.223544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576ffa-eb43-432b-87c6-2d6d4a74af92 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:07.234360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.235602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.236181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.237078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.262228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.269509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.303765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 985/1934] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:07.305732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.326057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319c770-959c-4f21-8e4a-0c2d7706d237 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:07.326419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90d3d54-a573-4bd8-8720-a45eb62aa98c service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:07.335811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.336433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.336854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.337517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.337733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.337998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.338247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.338788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] PATH_INFO: `/v3/projects/83032447af984380a4fd6bffebe852bc` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.341120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=83032447af984380a4fd6bffebe852bc)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.342947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.346937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.501641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:07.515934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.518073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dca0fb7-958f-4fdb-9a10-1803c7004775 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.521760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83032447af984380a4fd6bffebe852bc', 'request_id': 'req-f77a5eea-943c-423b-9f60-5338a2623dff'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.559766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 986/1935] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:07.562034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83032447af984380a4fd6bffebe852bc', 'request_id': 'req-f77a5eea-943c-423b-9f60-5338a2623dff'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.590068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83032447af984380a4fd6bffebe852bc', 'request_id': 'req-f77a5eea-943c-423b-9f60-5338a2623dff'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.632331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] The token cache is being invalidate because project 83032447af984380a4fd6bffebe852bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:07.633203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.635313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f77a5eea-943c-423b-9f60-5338a2623dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.658981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 950/1936] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:07 2022] DELETE /identity/v3/projects/83032447af984380a4fd6bffebe852bc => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:07.669469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.672371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.673739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.755344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:07.756633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.859554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:07.985414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 987/1937] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:07.992766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.993083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.993358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.036201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:08.037504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.068020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 951/1938] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:08.074488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.098869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca0ea-d05c-4f92-b6ec-83a04825c117 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.112056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f46308-e701-4494-b244-35e999042094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.112056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f46308-e701-4494-b244-35e999042094 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.112437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f46308-e701-4494-b244-35e999042094 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.112831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f46308-e701-4494-b244-35e999042094 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.114761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f46308-e701-4494-b244-35e999042094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:08.115257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f46308-e701-4494-b244-35e999042094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.117032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f46308-e701-4494-b244-35e999042094 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.156340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.159636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.182860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 988/1939] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:08.188294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f46308-e701-4494-b244-35e999042094 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.209443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70b510-1548-4b3d-9aca-4d94205fa408 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.230620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f46308-e701-4494-b244-35e999042094 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.231685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.236382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.237711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.238605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] PATH_INFO: `/v3/projects/8eb716b20b744990a62a8bd2e023d11c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.244035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb716b20b744990a62a8bd2e023d11c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.247163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.248146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.248599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.249107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.251282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.253594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.255629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.372384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 989/1940] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:08.389871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.487667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb716b20b744990a62a8bd2e023d11c', 'request_id': 'req-44eacd95-36b4-49ea-890b-1f822caaafb1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.498595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb716b20b744990a62a8bd2e023d11c', 'request_id': 'req-44eacd95-36b4-49ea-890b-1f822caaafb1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.513129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb716b20b744990a62a8bd2e023d11c', 'request_id': 'req-44eacd95-36b4-49ea-890b-1f822caaafb1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.543401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] The token cache is being invalidate because project 8eb716b20b744990a62a8bd2e023d11c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:08.543723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.544437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.548303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 952/1941] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:08 2022] DELETE /identity/v3/projects/8eb716b20b744990a62a8bd2e023d11c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:08.552566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.554490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.607220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c21c01-0cbf-4570-b04e-7eab78e902ab admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.701736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eacd95-36b4-49ea-890b-1f822caaafb1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.711534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.712418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.712749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.713131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] PATH_INFO: `/v3/users/bd66195067b8426c8427a20c0a7aac63` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.714482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd66195067b8426c8427a20c0a7aac63)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.716967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.757951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.758863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.759298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.759753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.760634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.763970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.806972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 990/1942] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/users => generated 289 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:08.808568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.810280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.826021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352e961-0952-44de-9cd6-3afaf5676155 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.827740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd66195067b8426c8427a20c0a7aac63'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.830279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:08.831651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.832534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.833432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.833432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.834067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.836084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.842812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd66195067b8426c8427a20c0a7aac63'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.843840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd66195067b8426c8427a20c0a7aac63'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.844617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd66195067b8426c8427a20c0a7aac63'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.850186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 991/1943] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:08.854443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.856733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.866591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd66195067b8426c8427a20c0a7aac63'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.876819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894efe73-bed9-4e38-806c-9a76f18ae80a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.889364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.891664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 953/1944] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:08 2022] DELETE /identity/v3/users/bd66195067b8426c8427a20c0a7aac63 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:08.891790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.891790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.892114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.892790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.903869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83b869-d32a-48ec-8c01-5b1e1e43f4cf admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.938380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.945878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 992/1945] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:09.053752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da965a2e-58eb-408d-a691-26345f4ac7c6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.066756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.080113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.081441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.082223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] PATH_INFO: `/v3/projects/ab5e8d79fd244c799adf83510ff86ef9/users/3dd89503e53041fd9ee563b1929af56f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.119984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab5e8d79fd244c799adf83510ff86ef9, user_id=3dd89503e53041fd9ee563b1929af56f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.126619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.149771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 954/1946] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:08 2022] PUT /identity/v3/projects/ab5e8d79fd244c799adf83510ff86ef9/users/3dd89503e53041fd9ee563b1929af56f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:09.161082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.161477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.161713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.211310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.229432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] MFA Rules not processed for user `3dd89503e53041fd9ee563b1929af56f`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:09.232645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.403482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 993/1947] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:09.416281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.418544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.448185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.450402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.600043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f71623-c8e3-4dc8-a9c6-aa8199a6518d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.607212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.609286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.610323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.611107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.613697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0f635-0de4-404c-ba7f-572782c6e3e3 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.619157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.620709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.621083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.621353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] PATH_INFO: `/v3/projects/ef787d9f64da430b8575dc3b8c98954a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.623336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef787d9f64da430b8575dc3b8c98954a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.627157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.635453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.641397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e965cbd-57a4-4c81-b30c-abf0e02caf5d service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.692482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 955/1948] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:09.790916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef787d9f64da430b8575dc3b8c98954a', 'request_id': 'req-ac808da2-9147-4a77-b99a-f20f6d264fae'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:09.807313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef787d9f64da430b8575dc3b8c98954a', 'request_id': 'req-ac808da2-9147-4a77-b99a-f20f6d264fae'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:09.829228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef787d9f64da430b8575dc3b8c98954a', 'request_id': 'req-ac808da2-9147-4a77-b99a-f20f6d264fae'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:09.863869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] The token cache is being invalidate because project ef787d9f64da430b8575dc3b8c98954a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:09.864182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:09.864872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ac808da2-9147-4a77-b99a-f20f6d264fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:09.870369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 994/1949] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:09 2022] DELETE /identity/v3/projects/ef787d9f64da430b8575dc3b8c98954a => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:09.873862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7775-9309-4290-a03d-96ee21565304 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.873862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7775-9309-4290-a03d-96ee21565304 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.874775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b7775-9309-4290-a03d-96ee21565304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.966320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-951b7775-9309-4290-a03d-96ee21565304 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:09.968069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-951b7775-9309-4290-a03d-96ee21565304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.060698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:10.124319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 956/1950] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:10.134144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.134588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.139777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.158869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:10.160261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.183393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 995/1951] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:10.183997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b7775-9309-4290-a03d-96ee21565304 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.196435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b7775-9309-4290-a03d-96ee21565304 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.210184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.210968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.211308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.211663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.212560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:10.213057 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.215294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.240959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 957/1952] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:10.242867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.253816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36da64a-2254-419d-8d70-b75180832c2a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.262092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.262846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.263126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.263544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.264077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.266277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.305940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 996/1953] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:10.311656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.332409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9077f4-f766-47ca-a603-7105c9fa9fc1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.454489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.455258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.455763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.456673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.456875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.459485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.487696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 958/1954] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/users => generated 283 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:10.492789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.506489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:10.516578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cdaa2-625b-4a8a-b059-50f425042f25 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.527424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.528706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.529878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.530468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.531616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.534140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.589289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 997/1955] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:10.594502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.610054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991c7dc-48fb-4d38-b0ce-a8908764747b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.624345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.625083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.625386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.625640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.626652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.633952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb772a32-15b6-4b70-aa9f-17ee16de5a96 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.661532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 959/1956] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:10.668093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.684331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd309e-e79c-4106-8a77-28a174990de3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.694958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.696005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.696344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.697092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] PATH_INFO: `/v3/projects/21a2c048f91649ddbb2bc94ff8bfce10/users/26d22c8f0d5e40adbe80c11c89748d0e/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.729907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a2c048f91649ddbb2bc94ff8bfce10, user_id=26d22c8f0d5e40adbe80c11c89748d0e, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.732795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.759897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 998/1957] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:10 2022] PUT /identity/v3/projects/21a2c048f91649ddbb2bc94ff8bfce10/users/26d22c8f0d5e40adbe80c11c89748d0e/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:10.778721 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.779045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.779559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.893639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] MFA Rules not processed for user `26d22c8f0d5e40adbe80c11c89748d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:10.896091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.980988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 960/1958] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:10.985239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.083416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb08bf2-c4fd-46ff-b053-2997b7cfa959 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.089873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.090629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.090919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.091189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.092042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.094512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.138056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.139888 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 999/1959] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:11.166996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3359c9-bc3b-4a24-9efa-cde024e448cd None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.198584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:11.237692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.238345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.238619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.239206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.239846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.242063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.267103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 961/1960] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:11 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:11.271952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.286665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82814d3-17e8-4bb0-958d-ac58178e60eb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.293969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.294849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.295167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.295619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.296520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.298719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.312053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1000/1961] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:11.317507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.343292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc234e-d03d-4b88-8cbe-e310298cdc5b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.359296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.360418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.360938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.361341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] PATH_INFO: `/v3/projects/b997444156654fd98b82cfc724577478/users/f650b698180d4752b36e100365885d44/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.387114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b997444156654fd98b82cfc724577478, user_id=f650b698180d4752b36e100365885d44, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.390700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.427130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 962/1962] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:11 2022] PUT /identity/v3/projects/b997444156654fd98b82cfc724577478/users/f650b698180d4752b36e100365885d44/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:11.434161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.456479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e64b4a-d2be-40ad-a46b-f426fd5bd80f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.535283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.535938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.536214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.536489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.537204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.539006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32838d09-73dc-4ade-9afb-36d43d9a57fb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.551112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1001/1963] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:11.555810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.567298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd594d00-d93d-428d-aa9b-d65a9ab80e6d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.572306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.573028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.573305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.573576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] PATH_INFO: `/v3/projects/b997444156654fd98b82cfc724577478/users/f650b698180d4752b36e100365885d44/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.575529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b997444156654fd98b82cfc724577478, user_id=f650b698180d4752b36e100365885d44, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.578508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.600538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 963/1964] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:11 2022] PUT /identity/v3/projects/b997444156654fd98b82cfc724577478/users/f650b698180d4752b36e100365885d44/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:11.611988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.611988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.611988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.661376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] MFA Rules not processed for user `f650b698180d4752b36e100365885d44`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:11.662462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.701089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:11.791800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1002/1965] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:11.814713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.816509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.897705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f05af7e-56f3-4cd2-a27a-c482fd9d8dd1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.907585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.908385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.908660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.908955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.945052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.946732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.960630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 964/1966] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:12.025656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.026909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.192859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.202785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627d608-4814-48a8-9dfa-caea6011b589 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.216144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.216835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.217099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.217372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.219944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d3843-3c42-4f56-842a-22ffc3f70b6a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.236669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.237505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.237883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.238252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] PATH_INFO: `/v3/users/642ab4f52a3b47fe8ffa46170bb393e3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.239911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] RBAC: Authorizing `identity:delete_user(user_id=642ab4f52a3b47fe8ffa46170bb393e3)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.242578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.370474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642ab4f52a3b47fe8ffa46170bb393e3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.392094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.392285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642ab4f52a3b47fe8ffa46170bb393e3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.394068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642ab4f52a3b47fe8ffa46170bb393e3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.394952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642ab4f52a3b47fe8ffa46170bb393e3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.396349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9eec92-b67e-4e4b-8d42-5625d34caf4d service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.409603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642ab4f52a3b47fe8ffa46170bb393e3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.443979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 965/1967] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:12.478705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1003/1968] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:12 2022] DELETE /identity/v3/users/642ab4f52a3b47fe8ffa46170bb393e3 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:12.485380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.485977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.486645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.526625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:12.527973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.591143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:12.724863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 966/1969] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:12.752766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.754692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.847499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:12.912316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8186d-22c9-4c10-a400-f33d74e3debf admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.950013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.952211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.953875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.954270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.981589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.983983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5d0c7d-bee2-4400-aec5-ae920a45045d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.019950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1004/1970] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:13.280256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.283604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.316327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92eabc-af05-4bfb-9c07-5049c614e731 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.333817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.335933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.337018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.337171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] PATH_INFO: `/v3/users/114aa2e38c5d4580890316f28ce308a4` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.340454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=114aa2e38c5d4580890316f28ce308a4)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.346068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.457263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114aa2e38c5d4580890316f28ce308a4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.486369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114aa2e38c5d4580890316f28ce308a4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.487411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114aa2e38c5d4580890316f28ce308a4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.489309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114aa2e38c5d4580890316f28ce308a4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.556372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114aa2e38c5d4580890316f28ce308a4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.594259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 967/1971] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:13 2022] DELETE /identity/v3/users/114aa2e38c5d4580890316f28ce308a4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:13.599489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.599816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.600297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.653973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.659400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.855865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1005/1972] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:13.865830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.867492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.885412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.887584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.890660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65ef55-a8e9-48c7-9b3d-1d46643386ab admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.913642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.914710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.914996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.915270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] PATH_INFO: `/v3/projects/aec86426074249009e3f7f8c246f3744` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.919360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec86426074249009e3f7f8c246f3744)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.938322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.030022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9ee4b-9b77-48e0-9ee5-23b6cf674075 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.043730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.045179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.045352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.045809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.064696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.066377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69dce-e5f6-43a0-bb72-841449234deb service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.092945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1006/1973] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:14.136473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec86426074249009e3f7f8c246f3744', 'request_id': 'req-100826e2-78f7-4dee-9528-29abd7af9860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.163105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec86426074249009e3f7f8c246f3744', 'request_id': 'req-100826e2-78f7-4dee-9528-29abd7af9860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.194573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec86426074249009e3f7f8c246f3744', 'request_id': 'req-100826e2-78f7-4dee-9528-29abd7af9860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.240197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] The token cache is being invalidate because project aec86426074249009e3f7f8c246f3744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:14.240535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.242265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-100826e2-78f7-4dee-9528-29abd7af9860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.249072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 968/1974] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:13 2022] DELETE /identity/v3/projects/aec86426074249009e3f7f8c246f3744 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:14.251464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.252263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.252791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.290946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:14.292356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.434038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1007/1975] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:14.443938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.444990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.445407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.480968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:14.482339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.499663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.500262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 969/1976] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:14.510731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ce8c7-2f30-4539-8ea8-841ea24913b9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.516139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.516835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.517229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.517502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.518863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:14.518863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.520134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.536583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1008/1977] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:14.541846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.557573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437034f-dfa6-458a-9431-c781ff21d76d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.567964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.568705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.569075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.569363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.570122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.572557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.580112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:14.585835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.597091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.630620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 970/1978] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/projects => generated 367 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:14.631865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.650055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23117826-7bcc-49e2-b273-3edd35c10348 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.671955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:14.736206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425a6768-fa33-42b2-96e2-75683d5c1d2d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.738662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.739321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.739599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.739869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] PATH_INFO: `/v3/projects/a6ad95d6dd384b82becda0cae061ec7d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.741109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ad95d6dd384b82becda0cae061ec7d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.742012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.742595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.742968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.743136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.743396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.743772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.745793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.773201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 971/1979] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:14.778251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.790200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ae04f-48df-42a6-b764-daaf1e8a1565 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.811335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ad95d6dd384b82becda0cae061ec7d', 'request_id': 'req-ea553396-cfa7-4464-896f-e97274cdc9a0'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.819795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ad95d6dd384b82becda0cae061ec7d', 'request_id': 'req-ea553396-cfa7-4464-896f-e97274cdc9a0'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.833130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ad95d6dd384b82becda0cae061ec7d', 'request_id': 'req-ea553396-cfa7-4464-896f-e97274cdc9a0'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.845482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] The token cache is being invalidate because project a6ad95d6dd384b82becda0cae061ec7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:14.845796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.846792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:14.848214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1009/1980] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:14 2022] DELETE /identity/v3/projects/a6ad95d6dd384b82becda0cae061ec7d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:14.851987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.853440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.869163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.869793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.870709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.870709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.870983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.872599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.884657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 972/1981] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:14.889327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.893491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.907237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c85422-4cce-4255-9640-e3374be95dd0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.915128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.915986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.916265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.916532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.917246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.918882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.933213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 973/1982] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:14.938443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea553396-cfa7-4464-896f-e97274cdc9a0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.939413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.947157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.947933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.948252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.949080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] PATH_INFO: `/v3/users/b1ad77c9274d41da932c486c223295ff` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.949955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1ad77c9274d41da932c486c223295ff)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.952162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.958363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bd6f73-bd7c-4578-a34b-0c2e5be305bb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.964255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.965491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.965491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.965711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] PATH_INFO: `/v3/projects/1db953a629a944a8b73a3721b461dc94/users/b1dab0ff34a34cd9af03a2d5edf5dcb6/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.981596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1db953a629a944a8b73a3721b461dc94, user_id=b1dab0ff34a34cd9af03a2d5edf5dcb6, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.984420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c98345e-e20a-442d-b209-345f6d51603c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.010125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 974/1983] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:14 2022] PUT /identity/v3/projects/1db953a629a944a8b73a3721b461dc94/users/b1dab0ff34a34cd9af03a2d5edf5dcb6/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:15.011964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.012251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.012531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.021967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1ad77c9274d41da932c486c223295ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.024430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.042130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1ad77c9274d41da932c486c223295ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.042758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1ad77c9274d41da932c486c223295ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.043830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1ad77c9274d41da932c486c223295ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.044622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] MFA Rules not processed for user `b1dab0ff34a34cd9af03a2d5edf5dcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:15.046202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.054833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1ad77c9274d41da932c486c223295ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.081999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1010/1984] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:14 2022] DELETE /identity/v3/users/b1ad77c9274d41da932c486c223295ff => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:15.095145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.171566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 975/1985] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:15.207285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.213876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.287184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.291211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.368099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94be7b-6651-4cb6-a884-9f61f7b7210c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.372313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.372956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.373366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.373496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.482150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54afe3-7696-43f2-8d3a-3b3ad83825b8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.497303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.500146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.513346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1011/1986] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:15.630224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.631914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.637973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.638634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.638914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.639183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] PATH_INFO: `/v3/users/26d22c8f0d5e40adbe80c11c89748d0e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.643042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26d22c8f0d5e40adbe80c11c89748d0e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.651856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.662061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9714c-b3e2-4ed0-b36c-201dfeb9bf29 service glance] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.687603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.688840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.689471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.690878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] PATH_INFO: `/v3/projects/fec8b3fda13e473bb07a96d02417bd38` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.691816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec8b3fda13e473bb07a96d02417bd38)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.694050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.732674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26d22c8f0d5e40adbe80c11c89748d0e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.744619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26d22c8f0d5e40adbe80c11c89748d0e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.745667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26d22c8f0d5e40adbe80c11c89748d0e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.747132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26d22c8f0d5e40adbe80c11c89748d0e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.771037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ed88d9c9-5a13-4381-a805-c3d48192d745 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26d22c8f0d5e40adbe80c11c89748d0e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.778348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.800500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 976/1987] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:15 2022] DELETE /identity/v3/users/26d22c8f0d5e40adbe80c11c89748d0e => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:15.808921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.809497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.810104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.813709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec8b3fda13e473bb07a96d02417bd38', 'request_id': 'req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.821660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec8b3fda13e473bb07a96d02417bd38', 'request_id': 'req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.832262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec8b3fda13e473bb07a96d02417bd38', 'request_id': 'req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.839330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:15.841214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.855242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] The token cache is being invalidate because project fec8b3fda13e473bb07a96d02417bd38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:15.855940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.856774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4da2ae10-2a03-499c-ad5e-cf0fab45ac17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.859582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1012/1988] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:15 2022] DELETE /identity/v3/projects/fec8b3fda13e473bb07a96d02417bd38 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:15.865529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2a93-e841-449b-b3e8-995ace122234 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.866025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2a93-e841-449b-b3e8-995ace122234 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.866761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2a93-e841-449b-b3e8-995ace122234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.882286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.910296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d21b2a93-e841-449b-b3e8-995ace122234 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:15.911918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d21b2a93-e841-449b-b3e8-995ace122234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.031663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 977/1989] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.043771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.045565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.102479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1013/1990] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.104862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.105154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.105435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.123425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58e681-d5eb-4b2a-9c61-d6764f0b79db None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.127654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.128299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.128572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.128838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.144675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.145804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.147079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.147702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.162899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1014/1991] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.163936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.166697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 978/1992] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:16.177901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5cff5-1ca1-4b01-9790-e3e878bc5463 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.184181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.185159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.185656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.186163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.187613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:16.188274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.190603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.213249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1015/1993] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:16.214618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.229830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71503-e764-4b8c-bd9c-43205388d0b5 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.238238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.238987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.239528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.240286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.241792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.245760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.283965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 979/1994] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:16.288237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.302256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff0e1a-60a0-4c96-8e34-b1cd30dc5ee6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.437509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.442770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.442770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.442770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.442770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.445569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.475452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1016/1995] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:16.483114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.497897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8247e1-e02f-481a-af58-6530b79f2166 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.511830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.512255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.512495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.512845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.513941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.515630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.532296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 980/1996] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:16.533302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.548054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd43b5-6e49-48f1-9397-0cf834bc74cd admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.554760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.555688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.556205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.556511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.557227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.560130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.573066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1017/1997] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:16.582863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.584316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.585958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.600586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe84d17c-1619-4743-b740-0c5ec4ad5bef admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.606180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0dd51-31c2-45fd-8502-95dfa42d3eac admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.616655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.617818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.618437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.618884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] PATH_INFO: `/v3/projects/3fa90c542632482a88d381457bac15b3/users/a78370f6a6fa4325ac2abce3b0791bc8/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.620357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.621572 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.622005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.623491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] PATH_INFO: `/v3/projects/21a2c048f91649ddbb2bc94ff8bfce10` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.625239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a2c048f91649ddbb2bc94ff8bfce10)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.631276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.656751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa90c542632482a88d381457bac15b3, user_id=a78370f6a6fa4325ac2abce3b0791bc8, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.662111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab2e8f-62ad-4d98-af3d-cf277531197d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.686721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1018/1998] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:16 2022] PUT /identity/v3/projects/3fa90c542632482a88d381457bac15b3/users/a78370f6a6fa4325ac2abce3b0791bc8/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:16.703750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.704227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.704662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.739403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a2c048f91649ddbb2bc94ff8bfce10', 'request_id': 'req-08608d51-98e5-4e77-a4d4-adce7180ba8f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.751018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a2c048f91649ddbb2bc94ff8bfce10', 'request_id': 'req-08608d51-98e5-4e77-a4d4-adce7180ba8f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.757335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:16.761419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] MFA Rules not processed for user `a78370f6a6fa4325ac2abce3b0791bc8`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.763707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.764761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a2c048f91649ddbb2bc94ff8bfce10', 'request_id': 'req-08608d51-98e5-4e77-a4d4-adce7180ba8f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.781584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] The token cache is being invalidate because project 21a2c048f91649ddbb2bc94ff8bfce10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:16.781878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.782562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.784761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 981/1999] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:16 2022] DELETE /identity/v3/projects/21a2c048f91649ddbb2bc94ff8bfce10 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:16.790296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.792140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.817692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:16.893080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1019/2000] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.900421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.901929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.918453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08608d51-98e5-4e77-a4d4-adce7180ba8f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.924448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.925150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.925431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.925706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] PATH_INFO: `/v3/users/f650b698180d4752b36e100365885d44` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.926981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f650b698180d4752b36e100365885d44)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.929061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.048663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f650b698180d4752b36e100365885d44'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.052703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79c673-d8f3-43a6-80e6-f0392b10785b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.069058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.069058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.069058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.069058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.069498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.069498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.079019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f650b698180d4752b36e100365885d44'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.079779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f650b698180d4752b36e100365885d44'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.080416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f650b698180d4752b36e100365885d44'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.101691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f650b698180d4752b36e100365885d44'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.121297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1020/2001] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/projects => generated 385 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:17.125020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 982/2002] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:16 2022] DELETE /identity/v3/users/f650b698180d4752b36e100365885d44 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:17.125788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.127529 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.279516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24121c48-bd7c-4409-8490-186fbd5d0959 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.287570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.288220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.288522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.288975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.290193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.292213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.313314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.331846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 983/2003] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:17.340349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.532688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b0677-2edf-40aa-96d7-57a475450847 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.557741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.558125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.558602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.559084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.560283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.563753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.588385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1021/2004] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:17.589997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.605584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3206bed-5d1a-42e6-87c3-ab127e657f50 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.613842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.614476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.614745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.615013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.615665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.617329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.630274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 984/2005] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:17.638474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.653318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00048b-f060-40a4-bb42-9a07d468e061 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.661676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.662580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.662859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.663131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] PATH_INFO: `/v3/projects/96e3c7b7d9314e169caf7082d13bea5c/users/8d99afc8a22f4ec7888e1acc74aa9d35/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.694236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e3c7b7d9314e169caf7082d13bea5c, user_id=8d99afc8a22f4ec7888e1acc74aa9d35, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.694363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.697903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adf0af1-0575-4432-876f-5a6ce555a36c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.698127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.725933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1022/2006] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:17 2022] PUT /identity/v3/projects/96e3c7b7d9314e169caf7082d13bea5c/users/8d99afc8a22f4ec7888e1acc74aa9d35/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:17.728282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0788d4-c6a3-45fe-a776-ac6e650a465b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.747247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.747739 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.747819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.786756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] MFA Rules not processed for user `8d99afc8a22f4ec7888e1acc74aa9d35`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:17.789395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.808832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.846656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.847472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.847763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.848029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] PATH_INFO: `/v3/projects/b997444156654fd98b82cfc724577478` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.849324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b997444156654fd98b82cfc724577478)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.851476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.882184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1023/2007] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:17.887909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.907796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf3863-ce3a-44f4-ae7c-6f2136a2689a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.922519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b997444156654fd98b82cfc724577478', 'request_id': 'req-d0a8e4ab-787e-4308-9806-6d0561853f9f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.931203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b997444156654fd98b82cfc724577478', 'request_id': 'req-d0a8e4ab-787e-4308-9806-6d0561853f9f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.943066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b997444156654fd98b82cfc724577478', 'request_id': 'req-d0a8e4ab-787e-4308-9806-6d0561853f9f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.957527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] The token cache is being invalidate because project b997444156654fd98b82cfc724577478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:17.957972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.958637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d0a8e4ab-787e-4308-9806-6d0561853f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.960234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 985/2008] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:17 2022] DELETE /identity/v3/projects/b997444156654fd98b82cfc724577478 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:17.967078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987e341-2cae-4714-8f0d-6b0950991d68 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.967338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987e341-2cae-4714-8f0d-6b0950991d68 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.967617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987e341-2cae-4714-8f0d-6b0950991d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.990383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a987e341-2cae-4714-8f0d-6b0950991d68 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:17.991638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a987e341-2cae-4714-8f0d-6b0950991d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.011945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.013662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.014004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.015756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.018465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.020497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.062873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1024/2009] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:18.068391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.070542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.091685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 986/2010] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:18.094243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.094243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.094680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.123964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.125861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.148035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.184166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bb7ec-3e8d-41a8-9879-d0414b1be7db admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.191720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.192551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.192955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.193324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.194114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.196945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.228554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1025/2011] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:18.233215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.244372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee3024-ab05-42c6-81bd-d6261a0752aa admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.250239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.254023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 987/2012] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:18.254210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.254210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.254210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.254210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.256581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.262976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.270855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc05fbe-3f16-4c60-9220-3cdace97a902 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.283724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.284289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1026/2013] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:18.284432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.284596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.284866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.285616 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:18.285906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.288129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.288234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.299161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbcb33-1886-45c3-a997-84e77add7a0e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.307427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.308547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.314152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.314488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] PATH_INFO: `/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f/users/e69271c5c851473b88c7a435d64bc001/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.316155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 988/2014] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:18.317192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.330327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17749b9-f1be-415c-ac2e-cf8a828aa0fc admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.336456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.337127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.337687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.337961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.338606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.340549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.346174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4be13f6ce424366a04a9d6d6787fe7f, user_id=e69271c5c851473b88c7a435d64bc001, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.349640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.352607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.372595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1027/2015] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:18 2022] PUT /identity/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f/users/e69271c5c851473b88c7a435d64bc001/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:18.379128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.394571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 989/2016] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:18.399865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.483137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66027596-49de-4352-a0ab-6a217b43b32e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.488932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.489570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.489852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.490122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.490933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.491842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5225137-1cd5-4730-9715-ec0a93cc0278 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.492543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.497524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.498273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.498613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.498945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.499590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.501942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.504565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1028/2017] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:18.508912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.528736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8bf8c-92ec-4c6d-8e31-b53c82bf7992 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.533643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.534497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.534688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 990/2018] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:18.534842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.535042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] PATH_INFO: `/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f/users/e69271c5c851473b88c7a435d64bc001/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.537259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4be13f6ce424366a04a9d6d6787fe7f, user_id=e69271c5c851473b88c7a435d64bc001, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.538846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.540204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b6a448-e371-4802-8811-d206af2e9129 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.550514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88125a-27d8-44df-b55b-f6a54d67fc10 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.553824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1029/2019] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:18 2022] PUT /identity/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f/users/e69271c5c851473b88c7a435d64bc001/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:18.562779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.562954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.563166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.629071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] MFA Rules not processed for user `e69271c5c851473b88c7a435d64bc001`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.636402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.681151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.683128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.683749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.684664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.687240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.696915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.715598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 991/2020] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:18.720168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.734730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a7b36-de1c-4894-9e74-a48c82c31d0a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.740670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.741674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.741980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.742245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.742933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.745004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.746982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.764469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 992/2021] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:18.776522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.830713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9eccbe-e8bb-4c4b-bde1-c7301048337b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.838607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.839309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.839584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.839861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] PATH_INFO: `/v3/projects/f2c89e9745d94b2cb447d4d404772ba5/users/9748b4b3d08e4d6fa6e6d25e4a64075c/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.847779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1030/2022] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.861639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.863739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c89e9745d94b2cb447d4d404772ba5, user_id=9748b4b3d08e4d6fa6e6d25e4a64075c, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.866385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.866708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303dea7-126b-44bb-a154-93451ebfd260 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.905568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 993/2023] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:18 2022] PUT /identity/v3/projects/f2c89e9745d94b2cb447d4d404772ba5/users/9748b4b3d08e4d6fa6e6d25e4a64075c/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:18.907915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.908328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.908675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.977223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] MFA Rules not processed for user `9748b4b3d08e4d6fa6e6d25e4a64075c`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.978385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a363b5-7d87-4727-88d4-e9464dde6731 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.979487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.007905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.065301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.066302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.066804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.067248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.122527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 994/2024] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:19.127383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.165818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.167582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.185250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1031/2025] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:19.223936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35a76f-d192-45c4-be81-a7d6275dbc6e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.229602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.230316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.230670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.231185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.232640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.235270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.300469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 995/2026] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:19.305998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.318151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771cb53d-64f7-46d5-b7f8-a22a14827d9d service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.431273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.432199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.432482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.437376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.438128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.440978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.480803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1032/2027] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:19.482054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.495500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba4364-8a14-451c-a8af-b955a464816e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.502415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.503599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.503933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.507050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.507050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.509143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.524326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 996/2028] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:19.529914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.541750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a9d269-1244-4502-9dd3-f384338f48d7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.552123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.553008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.553314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.553905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] PATH_INFO: `/v3/projects/63bbead4525d4c01a0ea3821bce466aa/users/d157713d8efa4b1c99870e94d64fa3da/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.594858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bbead4525d4c01a0ea3821bce466aa, user_id=d157713d8efa4b1c99870e94d64fa3da, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.598374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.619530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1033/2029] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:19 2022] PUT /identity/v3/projects/63bbead4525d4c01a0ea3821bce466aa/users/d157713d8efa4b1c99870e94d64fa3da/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:19.625080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.641381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811dd16-6e82-45b5-b9a6-20eabfbf7c93 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.701630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.753007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.754246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.754834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.755216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.757973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.758538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902fec3-0bb1-4160-801e-0cf280369c2f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.798451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 997/2030] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:19.805900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.833502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed17022f-80ad-4dee-a3bd-e467ee427296 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.842431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.843533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.843819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.844242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] PATH_INFO: `/v3/projects/63bbead4525d4c01a0ea3821bce466aa/users/d157713d8efa4b1c99870e94d64fa3da/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.846708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63bbead4525d4c01a0ea3821bce466aa, user_id=d157713d8efa4b1c99870e94d64fa3da, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.851208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.904781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1034/2031] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:19 2022] PUT /identity/v3/projects/63bbead4525d4c01a0ea3821bce466aa/users/d157713d8efa4b1c99870e94d64fa3da/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:19.924428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.924951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.925736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.997931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] MFA Rules not processed for user `d157713d8efa4b1c99870e94d64fa3da`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.999213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:20.174316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 998/2032] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:20.183239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:20.185075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:20.201547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:20.234632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:20.285269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f3afc-b233-431a-abe3-dfa0a790c7bb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:20.301215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:20.302364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:20.302874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:20.304463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:20.308180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:20.323930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0aeb5-6c34-4c6d-b094-5c7e9552c6a9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:20.332522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:20.333398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:20.333813 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:20.335309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:20.389776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:20.393160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:20.432665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1035/2033] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:20.537779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:20.539597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a7954e-1ebe-4776-8ced-3e2b1d012c77 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:20.555951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 999/2034] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:20.579580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:20.781685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964cc50-0fef-4b85-839f-f3e87ec9f72d service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:20.792180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:20.793447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:20.794315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:20.795132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] PATH_INFO: `/v3/users/9748b4b3d08e4d6fa6e6d25e4a64075c` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:20.797479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9748b4b3d08e4d6fa6e6d25e4a64075c)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:20.803017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:20.925513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9748b4b3d08e4d6fa6e6d25e4a64075c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:20.950298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9748b4b3d08e4d6fa6e6d25e4a64075c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:20.954030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9748b4b3d08e4d6fa6e6d25e4a64075c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:20.955245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9748b4b3d08e4d6fa6e6d25e4a64075c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:21.000145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9748b4b3d08e4d6fa6e6d25e4a64075c'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:21.080065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1036/2035] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:20 2022] DELETE /identity/v3/users/9748b4b3d08e4d6fa6e6d25e4a64075c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:21.090268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:21.091047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:21.091047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:21.147065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:21.149343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:21.167061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:21.290324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1000/2036] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:21.301309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:21.305192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:21.377248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:21.399914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4dfdd6-baf9-49e9-8a33-d33333cf3c63 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:21.406200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:21.407312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:21.407635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:21.407861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:21.420732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:21.422366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:21.441367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1037/2037] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:22.561256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:22.564101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:22.595186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81789-5320-4994-b8cb-7fca18e13358 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:22.603889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:22.605752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:22.606580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:22.607327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] PATH_INFO: `/v3/projects/f2c89e9745d94b2cb447d4d404772ba5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:22.611887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c89e9745d94b2cb447d4d404772ba5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:22.617793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:22.744731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c89e9745d94b2cb447d4d404772ba5', 'request_id': 'req-720315e4-3cc2-4c21-a178-d50807ba1568'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:22.753102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c89e9745d94b2cb447d4d404772ba5', 'request_id': 'req-720315e4-3cc2-4c21-a178-d50807ba1568'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:22.764002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c89e9745d94b2cb447d4d404772ba5', 'request_id': 'req-720315e4-3cc2-4c21-a178-d50807ba1568'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:22.794203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] The token cache is being invalidate because project f2c89e9745d94b2cb447d4d404772ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:22.794659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:22.795556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:22.797894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1001/2038] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:22 2022] DELETE /identity/v3/projects/f2c89e9745d94b2cb447d4d404772ba5 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:22.803935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:22.819405 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c7ea7-8473-48d3-b51e-aee0e9b97342 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:22.920935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:22.921930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:22.922356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:22.922798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] PATH_INFO: `/v3/users/d157713d8efa4b1c99870e94d64fa3da` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:22.924692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d157713d8efa4b1c99870e94d64fa3da)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:22.928125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:23.028971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd157713d8efa4b1c99870e94d64fa3da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:23.046396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd157713d8efa4b1c99870e94d64fa3da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:23.051877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd157713d8efa4b1c99870e94d64fa3da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:23.051877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd157713d8efa4b1c99870e94d64fa3da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:23.074492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8e1b83c9-5489-4d09-b5d1-4dfa08f739f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd157713d8efa4b1c99870e94d64fa3da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:23.133870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1038/2039] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:22 2022] DELETE /identity/v3/users/d157713d8efa4b1c99870e94d64fa3da => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:24.414696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.419829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.436303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720315e4-3cc2-4c21-a178-d50807ba1568 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:24.525561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:24.527047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.527047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.527047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] PATH_INFO: `/v3/projects/63bbead4525d4c01a0ea3821bce466aa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.528845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=63bbead4525d4c01a0ea3821bce466aa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:24.530321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:24.590682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:24.602545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63bbead4525d4c01a0ea3821bce466aa', 'request_id': 'req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:24.613694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63bbead4525d4c01a0ea3821bce466aa', 'request_id': 'req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:24.626164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63bbead4525d4c01a0ea3821bce466aa', 'request_id': 'req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:24.653620 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] The token cache is being invalidate because project 63bbead4525d4c01a0ea3821bce466aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:24.654338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:24.656275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b6aa5071-a1b4-4e85-b928-3cb0289cb32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:24.658964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1002/2040] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:24 2022] DELETE /identity/v3/projects/63bbead4525d4c01a0ea3821bce466aa => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:24.667312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.668720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.669244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.701103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:24.702512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.780784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1039/2041] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:24.792649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.793121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.793513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.821041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:24.822273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:24.834154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1003/2042] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:24.839170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.846411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:24.851113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec7856-c8cf-40a9-8963-2259cd1cb7c5 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:24.857510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:24.858360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.858942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.859336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.860734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:24.861464 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:24.863090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:24.877285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1040/2043] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:24.883287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.898647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d29b04-39ab-48ba-8a55-6a01531c1a37 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:24.904561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:24.905514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:24.905814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:24.906097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:24.907324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:24.911096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:24.943246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1004/2044] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:24.947939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:24.959477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373ff631-1b14-4afe-b2ff-ee22258c3310 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.043779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.044389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.044665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.044973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.045643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.047601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.076372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1041/2045] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:24 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:25.083832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.100236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80301369-2921-4ac5-ad8c-ee196fd6fccb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.106147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.106970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.107485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.107938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.108841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.110862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.132788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1005/2046] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:25.134798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.145777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787f8a4-e1cd-4b84-aa8d-e6367566d84a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.158943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.161219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.161219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.161586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.165961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.166460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f466a4ce-f350-4f9d-8d9d-ecc841d07be6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.185027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1042/2047] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:25.187316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.199010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7e9cb-6a3b-46ab-9446-8254ac0539c9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.204673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.205413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.205696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.205971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] PATH_INFO: `/v3/projects/bb33cc9642644f9c9fec6ce211164289/users/ddc94866a692407e8313d69c53adaab5/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.225608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb33cc9642644f9c9fec6ce211164289, user_id=ddc94866a692407e8313d69c53adaab5, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.229032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.263447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1006/2048] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:25 2022] PUT /identity/v3/projects/bb33cc9642644f9c9fec6ce211164289/users/ddc94866a692407e8313d69c53adaab5/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:25.265601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.265964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.266377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.313669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] MFA Rules not processed for user `ddc94866a692407e8313d69c53adaab5`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:25.315055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:25.400718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1043/2049] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:25.402044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.491358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.492862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:25.518276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b72ba-93b1-469c-b333-d88d4f9856f6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.526496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.528006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.528817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.529561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.531030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.535691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.567734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96cae5-31a5-48a8-b9cf-6ef80ca91e76 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.573075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.574079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.574516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.574910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.578317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:25.578878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1007/2050] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:25 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:25.584344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.603493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:25.670264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:25.681357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.683210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.691485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2375fd-dc02-44bc-95be-89c373f9957e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.696922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1044/2051] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:25.697658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.697658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.697935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.698219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.698998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.701107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.736753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1008/2052] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:25 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:25.738380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.752945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d3dca-564c-4eac-8375-3a24f1074a04 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.758337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.759085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.759457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.759906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.761089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.763453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.781965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1045/2053] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:25.787538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.802153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47b977-1820-47ee-bcca-0f73a8dfba84 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.807445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.808203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.808476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.808769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] PATH_INFO: `/v3/projects/b63876acc3b04e398847b192881bead0/users/3904e9bda0664b9ea04d63c8938e6362/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.828827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63876acc3b04e398847b192881bead0, user_id=3904e9bda0664b9ea04d63c8938e6362, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.832272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.855068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1009/2054] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:25 2022] PUT /identity/v3/projects/b63876acc3b04e398847b192881bead0/users/3904e9bda0664b9ea04d63c8938e6362/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:25.861465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:25.873170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22c7b0-51c3-4bc6-97d5-2f70a8466202 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:25.962613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:25.963422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:25.963780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:25.964143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:25.965303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:25.970107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce162e6a-653a-4985-acc8-dce66a8fde67 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:25.988689 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1046/2055] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:25.992461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:26.010086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbe7468-fda2-47b9-926d-67c0591e839f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:26.022750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:26.023709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:26.024105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:26.024507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] PATH_INFO: `/v3/projects/b63876acc3b04e398847b192881bead0/users/3904e9bda0664b9ea04d63c8938e6362/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:26.027495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63876acc3b04e398847b192881bead0, user_id=3904e9bda0664b9ea04d63c8938e6362, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:26.034307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:26.053252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1010/2056] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:25 2022] PUT /identity/v3/projects/b63876acc3b04e398847b192881bead0/users/3904e9bda0664b9ea04d63c8938e6362/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:26.070665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:26.071773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:26.071773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:26.115606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] MFA Rules not processed for user `3904e9bda0664b9ea04d63c8938e6362`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:26.117579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:26.216019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1047/2057] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:26.229657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:26.233442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:26.319273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2c0b6-a273-4b78-ba8b-e0fa3244bcf9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:26.333744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:26.334466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:26.334749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:26.335018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:26.347612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:26.349477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:26.363138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1011/2058] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:26.506699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:26.535277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0fd27-b62e-44b4-b093-c194ef1cd7f4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:26.551000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:26.551980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:26.552450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:26.553237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:26.605444 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:26.681926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:26.683494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21cca3-6903-48d7-a85e-f69989e2a2b6 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:26.703963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1048/2059] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:28.967311 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:28.970448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:29.095439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad605329-f05f-4220-ba15-9d46d2115a0d service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:29.100398 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:29.101102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:29.101386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:29.101664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] PATH_INFO: `/v3/users/a78370f6a6fa4325ac2abce3b0791bc8` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:29.102844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a78370f6a6fa4325ac2abce3b0791bc8)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:29.105007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:29.181328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:29.185555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a78370f6a6fa4325ac2abce3b0791bc8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:29.198606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a78370f6a6fa4325ac2abce3b0791bc8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:29.200418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a78370f6a6fa4325ac2abce3b0791bc8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:29.201583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a78370f6a6fa4325ac2abce3b0791bc8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:29.234478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a78370f6a6fa4325ac2abce3b0791bc8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:29.254217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1012/2060] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:28 2022] DELETE /identity/v3/users/a78370f6a6fa4325ac2abce3b0791bc8 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:29.259425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:29.260599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:29.261160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:29.287511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:29.289164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:29.404344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1049/2061] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:29.417619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:29.419110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:29.514845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463e6ab-1487-4c07-b743-e48e683ea680 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:29.519255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:29.520056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:29.520364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:29.520668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:29.532386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:29.534299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:29.550740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1013/2062] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:30.131132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:30.132561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:30.159820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd706fc-c271-40b3-905d-1c9013d5d610 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:30.167916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:30.168856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:30.169173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:30.169920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] PATH_INFO: `/v3/projects/3fa90c542632482a88d381457bac15b3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:30.172776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa90c542632482a88d381457bac15b3)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:30.174874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:30.277537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa90c542632482a88d381457bac15b3', 'request_id': 'req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.293226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa90c542632482a88d381457bac15b3', 'request_id': 'req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.308876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa90c542632482a88d381457bac15b3', 'request_id': 'req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.340598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] The token cache is being invalidate because project 3fa90c542632482a88d381457bac15b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:30.341171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.342034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bc24ba0e-4896-4fc6-ac3e-bc1609fd3ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.348162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1050/2063] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:30 2022] DELETE /identity/v3/projects/3fa90c542632482a88d381457bac15b3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:30.349467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:30.365175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077b76d-14fa-4305-905c-282250ddb33e service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:30.534363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:30.535839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:30.536352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:30.537531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] PATH_INFO: `/v3/users/8d99afc8a22f4ec7888e1acc74aa9d35` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:30.540254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d99afc8a22f4ec7888e1acc74aa9d35)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:30.543958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:30.668816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d99afc8a22f4ec7888e1acc74aa9d35'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.683154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d99afc8a22f4ec7888e1acc74aa9d35'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.689427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d99afc8a22f4ec7888e1acc74aa9d35'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.690907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d99afc8a22f4ec7888e1acc74aa9d35'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.705630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:30.724991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6ab4bc49-9ea7-4f31-a816-9ea1c93da05b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d99afc8a22f4ec7888e1acc74aa9d35'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:30.745521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1014/2064] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:30 2022] DELETE /identity/v3/users/8d99afc8a22f4ec7888e1acc74aa9d35 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:30.890631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:30.891295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:30.892248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:30.894905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1051/2065] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:51:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:30.904405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:30.904405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:30.904405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:30.939305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] MFA Rules not processed for user `d4895ca8a2854fb38ce08ce74b628671`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:30.941725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.084662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.092481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.184290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1015/2066] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:31.218040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.220108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.223828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:31.237699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63c5f45-203d-4b8b-8eb7-5018add591ad None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.247972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.248929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.249389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.255910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.274975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.276798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.282274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08747b0e-4436-41d3-9371-25fca9433ac9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.302335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.303537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.304232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.304992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] PATH_INFO: `/v3/users/f98fd1481406494e995df1128897415f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.307363 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f98fd1481406494e995df1128897415f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.310011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1016/2067] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:31.311724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.423029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f98fd1481406494e995df1128897415f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.438942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f98fd1481406494e995df1128897415f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.440043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f98fd1481406494e995df1128897415f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.441183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f98fd1481406494e995df1128897415f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.453452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f98fd1481406494e995df1128897415f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.464971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.467870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.493196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e5095-ae9b-4b4d-b7f9-d56e8d1a99c4 service placement] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.500911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.502607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.503262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.503897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] PATH_INFO: `/v3/projects/96e3c7b7d9314e169caf7082d13bea5c` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.505280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1052/2068] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/users/f98fd1481406494e995df1128897415f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:31.510599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e3c7b7d9314e169caf7082d13bea5c)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.514274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.621628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e3c7b7d9314e169caf7082d13bea5c', 'request_id': 'req-0d02336c-3dab-4d0a-95cf-ef134f81024d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.631469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e3c7b7d9314e169caf7082d13bea5c', 'request_id': 'req-0d02336c-3dab-4d0a-95cf-ef134f81024d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.645484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e3c7b7d9314e169caf7082d13bea5c', 'request_id': 'req-0d02336c-3dab-4d0a-95cf-ef134f81024d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.662275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] The token cache is being invalidate because project 96e3c7b7d9314e169caf7082d13bea5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:31.662653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.664184 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.666327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1017/2069] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/projects/96e3c7b7d9314e169caf7082d13bea5c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:31.673100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.678937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.756492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c8b95-6012-4881-a9f7-9ab2ffc1c762 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.811542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.813408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.853214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.853884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.854158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.854428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] PATH_INFO: `/v3/users/e69271c5c851473b88c7a435d64bc001` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.858077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69271c5c851473b88c7a435d64bc001)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.862490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.883893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:31.896133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:31.925451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d02336c-3dab-4d0a-95cf-ef134f81024d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.934855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.935957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.936408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.936874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] PATH_INFO: `/v3/projects/d86336ff2858481ea24231bfb10775f1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.939009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d86336ff2858481ea24231bfb10775f1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.942125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.942713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69271c5c851473b88c7a435d64bc001'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.954036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69271c5c851473b88c7a435d64bc001'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.964308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69271c5c851473b88c7a435d64bc001'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.968115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69271c5c851473b88c7a435d64bc001'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.984150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f3ec6817-e067-4907-96fd-0a38b9cb1a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69271c5c851473b88c7a435d64bc001'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.010612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1053/2070] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/users/e69271c5c851473b88c7a435d64bc001 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.043076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd86336ff2858481ea24231bfb10775f1', 'request_id': 'req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.055410 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd86336ff2858481ea24231bfb10775f1', 'request_id': 'req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.071734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd86336ff2858481ea24231bfb10775f1', 'request_id': 'req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.092950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] The token cache is being invalidate because project d86336ff2858481ea24231bfb10775f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:32.093354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.094534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9f5fb91a-805a-42b8-b556-fd1ed18b35fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.102925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1018/2071] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/projects/d86336ff2858481ea24231bfb10775f1 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.106138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.107097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.107527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.146959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.147843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.244211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1054/2072] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.258250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.258770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.259283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.284290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.285640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.302273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1019/2073] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.307894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.324921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad88a773-12b4-403a-aba0-7d623f68e069 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.336044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.337559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.339489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.340132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.341535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:32.342218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.345551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.365061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1055/2074] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:32.373779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.388990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63181c-eb04-4c7b-8e18-d6002be5920d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.397292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.398359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.398870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.399376 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.400621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.404646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.446490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1020/2075] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:32.452656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.467606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db254166-6705-4c64-a708-9212b7b84b9b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.478751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.482368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.552853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.553638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.553985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.554319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.555061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.557143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.572595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f481efc-5a5b-4851-bbf7-0ddf36de17c9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.578512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.579225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.579505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.579779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] PATH_INFO: `/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.581159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4be13f6ce424366a04a9d6d6787fe7f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.583603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.590853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1056/2076] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/users => generated 286 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:32.602920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.619970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7247404-6326-46e7-95ac-19da9fe9a27e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.628698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.629761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.630148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.630542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.631529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.634948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.639478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.658131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1057/2077] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:32.664085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.667770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4be13f6ce424366a04a9d6d6787fe7f', 'request_id': 'req-b249c876-26fe-4697-93ab-31c2a72f8976'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.671443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.679814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4be13f6ce424366a04a9d6d6787fe7f', 'request_id': 'req-b249c876-26fe-4697-93ab-31c2a72f8976'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.681582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6656970-3580-4ed5-a983-addc9b4ce0ed admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.694212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4be13f6ce424366a04a9d6d6787fe7f', 'request_id': 'req-b249c876-26fe-4697-93ab-31c2a72f8976'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.707912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] The token cache is being invalidate because project e4be13f6ce424366a04a9d6d6787fe7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:32.712297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.712297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b249c876-26fe-4697-93ab-31c2a72f8976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.712297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1021/2078] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:32 2022] DELETE /identity/v3/projects/e4be13f6ce424366a04a9d6d6787fe7f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.720788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b32b5-fa87-4609-b346-390f52a5b7dd None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.721290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b32b5-fa87-4609-b346-390f52a5b7dd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.721738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b32b5-fa87-4609-b346-390f52a5b7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.766947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-933b32b5-fa87-4609-b346-390f52a5b7dd None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.769163 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-933b32b5-fa87-4609-b346-390f52a5b7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.776294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.776588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.776863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.777240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.777965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.779761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.797206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1022/2079] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.801660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1058/2080] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:32.802809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.803157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.803497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.808352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.812568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.833103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84197ef-1436-400b-b701-03f9bb36db58 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.833294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.834948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.838940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.839688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.839980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.840254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] PATH_INFO: `/v3/projects/f18376d078a94034a751fa626a7f99be/users/d76be17fd480416595e26ba6d2f1a96d/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.843937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1023/2081] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:32.855968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.864375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18376d078a94034a751fa626a7f99be, user_id=d76be17fd480416595e26ba6d2f1a96d, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.867441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc41f74-903d-40e1-9015-b102edc6520a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.878663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ff080c-e112-44c0-995c-b059e4cf7b6d None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.885567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1059/2082] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:32 2022] PUT /identity/v3/projects/f18376d078a94034a751fa626a7f99be/users/d76be17fd480416595e26ba6d2f1a96d/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.897693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.898076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.898349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.941456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] MFA Rules not processed for user `d76be17fd480416595e26ba6d2f1a96d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.943042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.985926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.987009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.987390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.987877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.988706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:32.989255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.991507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.013071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1024/2083] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:33.013071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.015904 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.041803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f2baed-6539-4a91-a063-e78f1c316f49 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.049083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.050129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.050655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.051128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] PATH_INFO: `/v3/users/ddc94866a692407e8313d69c53adaab5` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.052939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc94866a692407e8313d69c53adaab5)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.055173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.064296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1060/2084] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:33.064296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.076232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dc542-3770-42c2-bcb5-5c972c22b9c7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.081298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.081929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.082209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.082491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.083241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.089166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.117484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc94866a692407e8313d69c53adaab5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.123138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1061/2085] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:33.123138 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.129721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.130797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc94866a692407e8313d69c53adaab5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.131530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc94866a692407e8313d69c53adaab5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.132332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc94866a692407e8313d69c53adaab5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.149792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc94866a692407e8313d69c53adaab5'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.171191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1025/2086] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:33 2022] DELETE /identity/v3/users/ddc94866a692407e8313d69c53adaab5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:33.172053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.175966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.220414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.270609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.274032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501184b-af4e-417a-92ea-dbf0d66f9621 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.343161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35032b5-3679-4b10-b313-3fb9f6c5c3b7 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.355223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.356260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.357106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.357564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.360138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.363443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902ce85-b96a-4dc3-81b4-797f63c12a83 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.399196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.400002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.400002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.401627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.402254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.434320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1026/2087] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/users => generated 280 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:33.435960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.436461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.437087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.464110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:33.465512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.473899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1027/2088] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:33.474822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.494084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.494412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f01f93-eee4-4b8e-9abb-8bea155b3146 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.497471 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.499250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.499850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.500127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.500396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.501177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.503449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.510843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1062/2089] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:33.515102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.518990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.535594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1028/2090] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:33.544878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.571600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121ffd0-18f8-4c93-8c20-5ffddfaac8c6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.577320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.577960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.578703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.578703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.579426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.581033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.596479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1029/2091] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:33.603112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.619694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf207b76-35ed-45e6-8c0b-5062a27289e6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.631375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.636697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.636697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.641846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] PATH_INFO: `/v3/projects/f82833c9d25a496282934358e1344d3e/users/43ae546cc4844d17908550d23ef0bd1f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.654476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1883d-395b-4781-a86d-d77209ced4b1 service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.658669 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.659406 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.659839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.660117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.679273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82833c9d25a496282934358e1344d3e, user_id=43ae546cc4844d17908550d23ef0bd1f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.685522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.687216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.688909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434ede3-a8a6-4d05-a530-f44a22550252 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.701663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1063/2092] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:33.726554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1030/2093] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:33 2022] PUT /identity/v3/projects/f82833c9d25a496282934358e1344d3e/users/43ae546cc4844d17908550d23ef0bd1f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:33.738218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.738534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.738811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.782226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] MFA Rules not processed for user `43ae546cc4844d17908550d23ef0bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:33.783923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.791356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.930889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1064/2094] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:34.086338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.088232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.173201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc6b6-8a27-49cc-8b31-b601a4c8f207 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.177542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.178170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.178446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.178719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.188195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.189624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.200246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1031/2095] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:34.221822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.257941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.259739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.309326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e9ee5-b921-4068-bce7-3a8fa6e0f630 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.317826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.318521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.318797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.319062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] PATH_INFO: `/v3/users/43ae546cc4844d17908550d23ef0bd1f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.320429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ae546cc4844d17908550d23ef0bd1f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.322990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.349187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f158e-852b-4acd-8fce-1ecd79c47ee8 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.354300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.354944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.355223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.355492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] PATH_INFO: `/v3/projects/bb33cc9642644f9c9fec6ce211164289` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.356637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb33cc9642644f9c9fec6ce211164289)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.359136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.380223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.395836 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ae546cc4844d17908550d23ef0bd1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.408866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ae546cc4844d17908550d23ef0bd1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.410670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ae546cc4844d17908550d23ef0bd1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.413950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ae546cc4844d17908550d23ef0bd1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.426819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-aac2d6de-10a8-4985-9eef-47c9ee5e6e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ae546cc4844d17908550d23ef0bd1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.433455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb33cc9642644f9c9fec6ce211164289', 'request_id': 'req-a18b336a-a73a-47e3-baa3-1a6d70409a32'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.441820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb33cc9642644f9c9fec6ce211164289', 'request_id': 'req-a18b336a-a73a-47e3-baa3-1a6d70409a32'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.458434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1065/2096] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/users/43ae546cc4844d17908550d23ef0bd1f => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:34.459081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb33cc9642644f9c9fec6ce211164289', 'request_id': 'req-a18b336a-a73a-47e3-baa3-1a6d70409a32'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.466218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.466751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.467249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.473204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] The token cache is being invalidate because project bb33cc9642644f9c9fec6ce211164289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:34.473911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.474467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.476808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1032/2097] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/projects/bb33cc9642644f9c9fec6ce211164289 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:34.480432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.482122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.501190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:34.503874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.539030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.578114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b336a-a73a-47e3-baa3-1a6d70409a32 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.614008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1066/2098] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:34.624012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.625596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.654265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.654918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.655192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.655458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] PATH_INFO: `/v3/users/3904e9bda0664b9ea04d63c8938e6362` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.656651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3904e9bda0664b9ea04d63c8938e6362)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.658667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.704389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca117783-514f-41db-87a2-7e29bd9bee1f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.708137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.709968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.710253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.711095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.711204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3904e9bda0664b9ea04d63c8938e6362'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.715643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.720203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3904e9bda0664b9ea04d63c8938e6362'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.721760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3904e9bda0664b9ea04d63c8938e6362'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.722647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3904e9bda0664b9ea04d63c8938e6362'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.730411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.732699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.742979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3904e9bda0664b9ea04d63c8938e6362'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.744255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1067/2099] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:34.758945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1033/2100] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/users/3904e9bda0664b9ea04d63c8938e6362 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.142959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.144692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.177690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9be05b-a423-4ebd-88d9-388e344d858c service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.228021 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.229432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.268847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.269660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.270391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.270574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] PATH_INFO: `/v3/users/3dd89503e53041fd9ee563b1929af56f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.272607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd89503e53041fd9ee563b1929af56f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.274809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.341328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b1f30-1d0d-488e-b690-8844a4b4cb6e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.346446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.347092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.347549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.347973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] PATH_INFO: `/v3/projects/f82833c9d25a496282934358e1344d3e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.349032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82833c9d25a496282934358e1344d3e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.350690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd89503e53041fd9ee563b1929af56f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.351055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.362951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd89503e53041fd9ee563b1929af56f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.363930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd89503e53041fd9ee563b1929af56f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.365083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd89503e53041fd9ee563b1929af56f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.376489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd89503e53041fd9ee563b1929af56f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.389973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.411411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1068/2101] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:35 2022] DELETE /identity/v3/users/3dd89503e53041fd9ee563b1929af56f => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.416087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.427837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.438345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82833c9d25a496282934358e1344d3e', 'request_id': 'req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.446579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82833c9d25a496282934358e1344d3e', 'request_id': 'req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.459918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82833c9d25a496282934358e1344d3e', 'request_id': 'req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.474115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] The token cache is being invalidate because project f82833c9d25a496282934358e1344d3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:35.474695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.475520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c0855c3-3c07-47e2-b9a2-c5d627100f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.477785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1034/2102] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:35 2022] DELETE /identity/v3/projects/f82833c9d25a496282934358e1344d3e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:35.479511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026be97-2c36-4946-97fd-6fe7a4e9f3df None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.479951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026be97-2c36-4946-97fd-6fe7a4e9f3df None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.480374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8026be97-2c36-4946-97fd-6fe7a4e9f3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.502282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8026be97-2c36-4946-97fd-6fe7a4e9f3df None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.503763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8026be97-2c36-4946-97fd-6fe7a4e9f3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.518839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9997af6-8db4-4c0e-a2bb-1ab2bfcbea80 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.566657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.583729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1035/2103] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:35.583729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.583729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.583729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.604240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.605691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.607989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.609079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.609779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.610495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] PATH_INFO: `/v3/projects/b63876acc3b04e398847b192881bead0` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.612741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63876acc3b04e398847b192881bead0)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.615168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1036/2104] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:35.616024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.616175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.617608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.710939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63876acc3b04e398847b192881bead0', 'request_id': 'req-6a53de8c-938d-46c7-aede-9fca65291b05'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.722699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63876acc3b04e398847b192881bead0', 'request_id': 'req-6a53de8c-938d-46c7-aede-9fca65291b05'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.731400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f848b2-b27b-4606-9c78-1ebe8b97989e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.733928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63876acc3b04e398847b192881bead0', 'request_id': 'req-6a53de8c-938d-46c7-aede-9fca65291b05'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.747645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] The token cache is being invalidate because project b63876acc3b04e398847b192881bead0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:35.748092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.748954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a53de8c-938d-46c7-aede-9fca65291b05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.751096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1069/2105] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:35 2022] DELETE /identity/v3/projects/b63876acc3b04e398847b192881bead0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.752674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd069f-e4cc-4047-9c26-bf363ff3af61 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.753108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd069f-e4cc-4047-9c26-bf363ff3af61 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.753490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd069f-e4cc-4047-9c26-bf363ff3af61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.778977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7cdd069f-e4cc-4047-9c26-bf363ff3af61 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.780241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7cdd069f-e4cc-4047-9c26-bf363ff3af61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.805211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.805833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.806160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.806469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.808659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.813757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.878197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1070/2106] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:35.879589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc46b1-254f-4836-a95a-dd143a789e51 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.879866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc46b1-254f-4836-a95a-dd143a789e51 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.880131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fc46b1-254f-4836-a95a-dd143a789e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.891912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.893603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.901115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a5fc46b1-254f-4836-a95a-dd143a789e51 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.902358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a5fc46b1-254f-4836-a95a-dd143a789e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.905192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1037/2107] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:35.906102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.912942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1071/2108] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:35.919349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c1c19-1592-4ff5-a079-9d4062223e08 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.921936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.922389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.922984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.925117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.925895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.926822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.926822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.927516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:35.927913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.929717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.942823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1038/2109] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:35.945538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.947117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.951515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.961150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1072/2110] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:35.962718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.970678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efafad-644f-4643-839a-ed105786d16f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.979820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1ec64-ca2c-45eb-bac7-8a8d07a3f148 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.980182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.980822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.981130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.981538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.982035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.984073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.986288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.987052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.987458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.987913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.988874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:35.989022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.990941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.006936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1073/2111] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:36.008216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.021350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76d57a-c2c2-4386-9497-8e62817e87ae admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.021827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1039/2112] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:36.031820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.099292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.138455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.139505 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.139753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.139988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.141071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.144127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.167092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6dc3e-e891-476c-8d4d-4ff7586d81b5 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.174920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.176085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.176537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.177900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.179656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.189999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.203374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1074/2113] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/projects => generated 381 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:36.209285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.228785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1040/2114] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:36.234612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.253913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acc348-bc9c-4c4c-9223-87fcc1bf5cfa admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.316565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bbdfab-6795-4cd2-853e-7893e01e0f22 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.326401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.327210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.327963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.328492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.329558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.333006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.339820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.340746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.341244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.341682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.342746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.345555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.364611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1041/2115] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:36.364611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.364611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.365627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1075/2116] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/users => generated 291 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:36.366486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.376876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a33470-f9cb-4c67-aa41-a53c756bf007 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.377125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6d448-1cc2-4be0-8a04-320c89cdbfa6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.381822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.382018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.382442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.382659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.382790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.382943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.383101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.383354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] PATH_INFO: `/v3/projects/ab5e8d79fd244c799adf83510ff86ef9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.383646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.384331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab5e8d79fd244c799adf83510ff86ef9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.385357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.386427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.396515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1076/2117] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:36.397936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.404437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.411804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50882a9-cde1-4985-ae8b-654622bb26ff admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.422369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.423609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.424211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.424782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.426034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.429051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.445802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab5e8d79fd244c799adf83510ff86ef9', 'request_id': 'req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.448347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1077/2118] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:36.448599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.456810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab5e8d79fd244c799adf83510ff86ef9', 'request_id': 'req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.463647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f5bc4-5bf1-4e48-8a75-3a00a7e008c3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.467042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab5e8d79fd244c799adf83510ff86ef9', 'request_id': 'req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.484058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] The token cache is being invalidate because project ab5e8d79fd244c799adf83510ff86ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:36.484471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.485321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.487238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1042/2119] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:36 2022] DELETE /identity/v3/projects/ab5e8d79fd244c799adf83510ff86ef9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:36.487933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.489485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.550542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.557541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.558228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.558525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.558905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] PATH_INFO: `/v3/projects/31e01eff8f1446a18497195dabfe58bf/users/903147b749ff4fc88e8e75b305c05efd/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.576679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e01eff8f1446a18497195dabfe58bf, user_id=903147b749ff4fc88e8e75b305c05efd, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.579741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5531d9-21f1-4125-8bec-befe0372a747 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.586544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71a9a64-9ca3-4267-8d1b-11f5d14b1e6a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.594027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.594759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.595056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.595280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.596067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.597614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1078/2120] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:36 2022] PUT /identity/v3/projects/31e01eff8f1446a18497195dabfe58bf/users/903147b749ff4fc88e8e75b305c05efd/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:36.597883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad5970-e707-4f64-8d4f-8d1a59f996da admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.598904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.599188 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.599459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.609994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1043/2121] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:36.611268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67beb8f-500e-4564-ab37-be988603ebf7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.611558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67beb8f-500e-4564-ab37-be988603ebf7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.611832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67beb8f-500e-4564-ab37-be988603ebf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.620375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:36.622038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.644740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d67beb8f-500e-4564-ab37-be988603ebf7 None None] MFA Rules not processed for user `903147b749ff4fc88e8e75b305c05efd`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:36.646023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d67beb8f-500e-4564-ab37-be988603ebf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.702593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1079/2122] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:36.703392 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.714048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098c473-2117-48c7-ac4d-556bd88ec374 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.727551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.728247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.728671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.728774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] PATH_INFO: `/v3/projects/17a95ff6870d40c6a94ee5243c78629d/users/b1f4802f4a3a49f8bfc3b1e2e83d7724/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.740821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1044/2123] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:36.742156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.742438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.742709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.744948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17a95ff6870d40c6a94ee5243c78629d, user_id=b1f4802f4a3a49f8bfc3b1e2e83d7724, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.747727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.762292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1080/2124] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:36 2022] PUT /identity/v3/projects/17a95ff6870d40c6a94ee5243c78629d/users/b1f4802f4a3a49f8bfc3b1e2e83d7724/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:36.771078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.771078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.771253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:36.771253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.823601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.835729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1045/2125] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:36.836676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.838076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.842229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32c544-ce9c-48fc-b9c1-1ca7a949df02 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.847443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.848082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.848360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.848628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] PATH_INFO: `/v3/users/d76be17fd480416595e26ba6d2f1a96d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.849852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76be17fd480416595e26ba6d2f1a96d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.850187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be913a7-e257-4715-b608-eda55673b3b8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.851779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.855118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.855730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.856006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.856275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.856950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.858861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d84cc7-e84c-46c4-b774-b38fdf74fa69 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.890356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1046/2126] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:36.894760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.895118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.895456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.912032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76be17fd480416595e26ba6d2f1a96d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.921972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76be17fd480416595e26ba6d2f1a96d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.922717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76be17fd480416595e26ba6d2f1a96d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.923391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76be17fd480416595e26ba6d2f1a96d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.928101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] MFA Rules not processed for user `b1f4802f4a3a49f8bfc3b1e2e83d7724`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:36.929353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.933374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76be17fd480416595e26ba6d2f1a96d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.956205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.957066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1081/2127] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:36 2022] DELETE /identity/v3/users/d76be17fd480416595e26ba6d2f1a96d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:36.958350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.960236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.001087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1047/2128] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.002421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.003906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.038952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772d4a0-1aa5-4d1d-91e0-9af8ad51ce1f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.043660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.044287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.044555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.044816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.045497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:37.045800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.047476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e9179-f49d-4356-a87f-9a23cbe6f24a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.061787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1082/2129] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:37.063113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.063398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.063662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.085162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.086521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.094572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1083/2130] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.095735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.097328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.101330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374e7a5-4295-4465-a034-095ab53d7f3a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.107444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.108367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.108632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.108913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.109549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.109889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbed388-31bd-4393-82da-fae74fdfb397 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.111667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.114768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.115375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.115667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.116069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.116700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.118722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.134591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1048/2131] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.135525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.146538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10307b50-ba99-4282-afd6-22d256761a1d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.149818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1084/2132] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.150683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.152196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.152823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.153175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.153420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.154067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.154196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.156017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.186836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1049/2133] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.188102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.199805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.221029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cb6bb-7738-4e9e-ab63-e0bb8025f620 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.255345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e0b0d-aca2-4469-a149-200c71ce269c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.260145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.260698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.261007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.261278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.261921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.263526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.271526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.272149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.272425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.272691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.273869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1050/2134] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.275122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.282939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.284393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.285420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7679ab-0777-4464-9658-ecf1c7bb907d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.291830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.292481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.292791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.293101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.293746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.294965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1085/2135] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:37.295710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.295820 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.305808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a6eb0e-cde1-4c7d-982a-fa6d421d967d service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.312819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.314248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.314542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.315260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.315915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.317894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.319760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1051/2136] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/users => generated 298 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.320647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.334986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e1c85-e326-4209-b14e-caeebe258c7a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.340728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.341482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.341659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.342088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.342803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.344564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.349326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1086/2137] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/users => generated 293 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.351229 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.362525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1052/2138] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.362525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.362799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64278a12-ea27-4105-997c-1ca29266d991 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.366432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.369784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.370455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.370915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.371070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.371612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.373804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.375222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8747a-b458-4cc9-9336-2a717eda8116 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.383658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.384360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.384736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.385105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.386171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.386546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1087/2139] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.387573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.388386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.398876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649fae1-ef39-40e5-880d-e643ccba5c03 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.404386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1053/2140] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.404386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.407607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.408459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.408817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.409182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] PATH_INFO: `/v3/projects/eeef3b7cf98a4e318f5e35b3a258cffa/users/4c4c23fe08f44db59d9fd1cb74701ece/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.415085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb813e86-da4d-4811-9efb-37c873b87849 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.419710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.420382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.420937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.420937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] PATH_INFO: `/v3/projects/06c057000a584260921baed6fa37ebe1/users/9895bdfb5b2c4871a137574121d4774f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.429604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeef3b7cf98a4e318f5e35b3a258cffa, user_id=4c4c23fe08f44db59d9fd1cb74701ece, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.432387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.438810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c057000a584260921baed6fa37ebe1, user_id=9895bdfb5b2c4871a137574121d4774f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.442394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.456164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1088/2141] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:37 2022] PUT /identity/v3/projects/eeef3b7cf98a4e318f5e35b3a258cffa/users/4c4c23fe08f44db59d9fd1cb74701ece/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.462270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1054/2142] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:37 2022] PUT /identity/v3/projects/06c057000a584260921baed6fa37ebe1/users/9895bdfb5b2c4871a137574121d4774f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.462833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.463508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.537119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd35fe-98fb-41b6-a1ab-2b7bf4e38db2 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.542575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.543441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.543821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.544238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.544921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.545053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf135087-6417-4229-bc1a-a63d89e3dbe5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.546791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ebae-544a-4707-8984-a8ca86a50cbd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.550611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.551298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.551728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.552006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.552653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.554397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.566460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1055/2143] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.567871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294b36-feea-4bd0-8003-904a02f8e71a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.568161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294b36-feea-4bd0-8003-904a02f8e71a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.568432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82294b36-feea-4bd0-8003-904a02f8e71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.570912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1089/2144] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.571865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.583412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465876c-67fb-4851-80e7-12ca082257d2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.590307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.591130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.591541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.591851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] PATH_INFO: `/v3/projects/06c057000a584260921baed6fa37ebe1/users/9895bdfb5b2c4871a137574121d4774f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.594564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c057000a584260921baed6fa37ebe1, user_id=9895bdfb5b2c4871a137574121d4774f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.608913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.614595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1090/2145] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:37 2022] PUT /identity/v3/projects/06c057000a584260921baed6fa37ebe1/users/9895bdfb5b2c4871a137574121d4774f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.617765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.622737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-82294b36-feea-4bd0-8003-904a02f8e71a None None] MFA Rules not processed for user `4c4c23fe08f44db59d9fd1cb74701ece`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.625033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-82294b36-feea-4bd0-8003-904a02f8e71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.651702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.696259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01969ca2-6492-486e-97b9-ec0b999a4804 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.701296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.702041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.702385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.702654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] PATH_INFO: `/v3/projects/eca4e96a9fdd4b25ae6aee54432f4e59/users/8c0549313b8f43098629b2a8b1451a4f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.706522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1056/2146] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.707797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.708155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.708475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.719588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eca4e96a9fdd4b25ae6aee54432f4e59, user_id=8c0549313b8f43098629b2a8b1451a4f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.722348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.736703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] MFA Rules not processed for user `9895bdfb5b2c4871a137574121d4774f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.738136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.738251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1091/2147] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:37 2022] PUT /identity/v3/projects/eca4e96a9fdd4b25ae6aee54432f4e59/users/8c0549313b8f43098629b2a8b1451a4f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.739082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.748845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.780306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.814347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535697c-72d2-4f86-833a-807da53b9945 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.817848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1057/2148] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.818743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.819296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.819941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.820205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.820918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] PATH_INFO: `/v3/projects/f18376d078a94034a751fa626a7f99be` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.821787 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18376d078a94034a751fa626a7f99be)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.823670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.830146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10f115-e4e9-43fd-81ad-5497c8d84070 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.834988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.835403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.835809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.836085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.836718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.838807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f442234-3ea1-41ad-a54a-885744e9460c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.874670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1058/2149] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.874858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18376d078a94034a751fa626a7f99be', 'request_id': 'req-9cae8b8b-c51e-477b-aa76-440d4cca9784'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:37.874957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.874957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.874957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.880033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18376d078a94034a751fa626a7f99be', 'request_id': 'req-9cae8b8b-c51e-477b-aa76-440d4cca9784'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:37.891909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18376d078a94034a751fa626a7f99be', 'request_id': 'req-9cae8b8b-c51e-477b-aa76-440d4cca9784'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:37.902928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] MFA Rules not processed for user `8c0549313b8f43098629b2a8b1451a4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.904196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.905527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] The token cache is being invalidate because project f18376d078a94034a751fa626a7f99be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:37.905822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:37.906518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:37.908298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1092/2150] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:37 2022] DELETE /identity/v3/projects/f18376d078a94034a751fa626a7f99be => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.909179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.911884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.956348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.976259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1059/2151] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.976769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cae8b8b-c51e-477b-aa76-440d4cca9784 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.977358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.978810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.033823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.033823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.033823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.033823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.034822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.053199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2155e3-b889-4efd-8ebe-0d1873473789 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.058188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.058819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.059667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.059977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.060650 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.063113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2928e3b-4a7b-4c54-b0ee-8a265de26961 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.088692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1060/2152] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:38.088692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.088692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.088692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.100484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.101974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.108594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:38.109914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.113404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1093/2153] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:38.114164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.115459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.121000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1061/2154] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:38.121953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.126794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21576e89-ff99-4f02-88f5-1e0fe7851f7b service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.131175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.132664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.132664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.132664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.135925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52def621-9f8e-4d4b-8b8f-09d669fc8505 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.142092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.142805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.143173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.143575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.143741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.144364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.145322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bae5dd-eb30-4a99-b560-d52bfb26e3ba service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.147709 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.156794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1094/2155] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:38.158216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.158497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.158816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.164925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1062/2156] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:38.167542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.178750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a336695-3fe7-4307-adb5-c03e09adbe24 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.183873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.184554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.184933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.185185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] PATH_INFO: `/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83/users/2a535dd0cfce43fcb4d2b616b3e28eb1/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.200999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:38.202504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.206519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e38fb52e9fb44a7b2d77788c2e88b83, user_id=2a535dd0cfce43fcb4d2b616b3e28eb1, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.209828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.213176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:38.216028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1095/2157] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:38.221279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.237010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1063/2158] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:38 2022] PUT /identity/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83/users/2a535dd0cfce43fcb4d2b616b3e28eb1/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:38.238351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.239495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aabd2f-5315-4764-a830-3bd326c61df6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.318503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a912c-66d7-43ab-b4d3-e77d9c6f525d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.322656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.323485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.324019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.324131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.324236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.324986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.325357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.325702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.326321 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.326473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:38.326877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.328737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.359364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1096/2159] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:38.359364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.373960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92c02b-878a-4bb1-bcbe-c880b57c91f5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.379438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.380065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.380706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.381096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.381957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.384104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.397160 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1097/2160] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:38.398123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.404935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.407157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.410544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc343716-8feb-4937-9f35-3cbb95a5db47 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.417288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.417918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.418199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.418476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.419210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.420638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1064/2161] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:38.422309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.422708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.428954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:38.439262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea56dd1-1418-48d2-94c2-91c1913c901b service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.445862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.446535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.446858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.447085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] PATH_INFO: `/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83/users/2a535dd0cfce43fcb4d2b616b3e28eb1/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.451142 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e38fb52e9fb44a7b2d77788c2e88b83, user_id=2a535dd0cfce43fcb4d2b616b3e28eb1, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.454302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08323c19-d53f-4c51-926f-6bf164f80e6e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.470767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1098/2162] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/projects => generated 377 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:38.471425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1065/2163] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:38 2022] PUT /identity/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83/users/2a535dd0cfce43fcb4d2b616b3e28eb1/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:38.472681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.482588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.482974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.483362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.520751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] MFA Rules not processed for user `2a535dd0cfce43fcb4d2b616b3e28eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:38.523219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.599508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb13770-8ae2-4221-9807-d97154052932 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.609993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.610640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.610924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.611202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.611963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.613982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.623997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1066/2164] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:38.650912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1099/2165] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:38.652711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.666085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f0c8f-81d5-4e06-9b49-1a81ee1291f4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.672166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.673059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.673344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.673622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.674287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.676280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.683462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.705131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1067/2166] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:38.706462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.717834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44567663-e02d-4f1f-ade2-7f1196d853b7 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.724378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.725754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.726040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.726482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.727304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.729047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.738242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:38.746192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1068/2167] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:38.750941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.762077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a316ca56-f9ae-43b6-9f7b-e8e3405a3e0d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.764192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e0702-4565-4ef1-81e4-292702275b64 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.770819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.770989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.771582 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.771714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.771879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.771994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.772145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.772244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] PATH_INFO: `/v3/projects/8e89402e388d47e7ba42747b2d4c10f2/users/fca0d685cb9a4603a85a0c5dd5eb8ab9/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.773938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.791984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e89402e388d47e7ba42747b2d4c10f2, user_id=fca0d685cb9a4603a85a0c5dd5eb8ab9, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.794938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.813441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1069/2168] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:38 2022] PUT /identity/v3/projects/8e89402e388d47e7ba42747b2d4c10f2/users/fca0d685cb9a4603a85a0c5dd5eb8ab9/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:38.814384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.816282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.851062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.852563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1cc95-966d-491b-ba96-97233b2eb17b service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.865833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1100/2169] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:38.867539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.867825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.868098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.900658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccade8-12fd-46c7-8074-913fb72b1191 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.908209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.909450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.909911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.910357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] PATH_INFO: `/v3/users/b1f4802f4a3a49f8bfc3b1e2e83d7724` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.912606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] MFA Rules not processed for user `fca0d685cb9a4603a85a0c5dd5eb8ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:38.913638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f4802f4a3a49f8bfc3b1e2e83d7724)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.915540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.916040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.942849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:38.983073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f4802f4a3a49f8bfc3b1e2e83d7724'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:38.994869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f4802f4a3a49f8bfc3b1e2e83d7724'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:38.996352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f4802f4a3a49f8bfc3b1e2e83d7724'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:38.997099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f4802f4a3a49f8bfc3b1e2e83d7724'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:39.010742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1101/2170] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:39.013559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-07f5e833-2cfd-464d-9932-b7b8d288099e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f4802f4a3a49f8bfc3b1e2e83d7724'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:39.015210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.029064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e2b1c-5a1b-4cb2-947f-5eda2e3d0d33 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.033866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1070/2171] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:38 2022] DELETE /identity/v3/users/b1f4802f4a3a49f8bfc3b1e2e83d7724 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:39.039581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.040344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.040549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.066992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:39.068315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.107139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.107774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.108073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.108440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.109028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.111105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.137206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:39.142238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1102/2172] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:39 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:39.146247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.148906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.160143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1071/2173] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:39.170922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.172592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.267801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6f3a9-7088-4853-b15d-6039af3538c4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.274840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09648aa6-a569-4ba6-91c1-2aba60d25573 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.292633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.293271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.293558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.293830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.294575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.296989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.321636 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1103/2174] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:39 2022] POST /identity/v3/users => generated 295 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:39.325962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.337675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c968903-699c-4006-bf5b-b195c0d202a2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.343237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.343854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.344178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.344486 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.345294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.347624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.347832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.347937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.347937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.348244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.359385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.360510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.362287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1104/2175] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:39.366732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.373360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1072/2176] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:39.377979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6792a-c1b5-4317-bd66-5b7da0a87994 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.383177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.384018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.384299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.384568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] PATH_INFO: `/v3/projects/c1ed373f55ca49939515fa87603eaa70/users/cdfa32e6ec8c49e589154e665ea6e70f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.403276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ed373f55ca49939515fa87603eaa70, user_id=cdfa32e6ec8c49e589154e665ea6e70f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.405915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.418485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:39.428481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1105/2177] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:39 2022] PUT /identity/v3/projects/c1ed373f55ca49939515fa87603eaa70/users/cdfa32e6ec8c49e589154e665ea6e70f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:39.432555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.445295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78741b40-cc2f-48ca-9cf8-84d974d12ed2 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.532343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.533014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.533292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.533564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.534232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.535900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22455b-853e-4a76-950c-b8b9fb9d9fc1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.553032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1073/2178] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:39.557586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.569629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44d5a0-10b7-4f59-aa06-68619a018605 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.575617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.576647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.577215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.577677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] PATH_INFO: `/v3/projects/c1ed373f55ca49939515fa87603eaa70/users/cdfa32e6ec8c49e589154e665ea6e70f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.580361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ed373f55ca49939515fa87603eaa70, user_id=cdfa32e6ec8c49e589154e665ea6e70f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.583402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.599855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1106/2179] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:39 2022] PUT /identity/v3/projects/c1ed373f55ca49939515fa87603eaa70/users/cdfa32e6ec8c49e589154e665ea6e70f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:39.610981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.613894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.613894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.649300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] MFA Rules not processed for user `cdfa32e6ec8c49e589154e665ea6e70f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:39.650603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.708575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:39.757374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1074/2180] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:39.777848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.779604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.869722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c29d0a-7697-4e38-9e7b-572ff6c0442a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:39.875267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:39.876618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:39.877273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:39.877623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:39.880305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.954566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:39.956161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:39.961462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:39.963627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:39.967142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1107/2181] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:40.046559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a016d91-50da-4bd0-a273-4888c7a2abf1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:40.052493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:40.053312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:40.053720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:40.054076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] PATH_INFO: `/v3/projects/17a95ff6870d40c6a94ee5243c78629d` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:40.055609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17a95ff6870d40c6a94ee5243c78629d)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:40.057768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:40.115589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17a95ff6870d40c6a94ee5243c78629d', 'request_id': 'req-cfa5dc17-1141-416e-8204-853aa8333690'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.124228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17a95ff6870d40c6a94ee5243c78629d', 'request_id': 'req-cfa5dc17-1141-416e-8204-853aa8333690'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.137803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17a95ff6870d40c6a94ee5243c78629d', 'request_id': 'req-cfa5dc17-1141-416e-8204-853aa8333690'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.150533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] The token cache is being invalidate because project 17a95ff6870d40c6a94ee5243c78629d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:40.150836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.153235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.155055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1075/2182] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:39 2022] DELETE /identity/v3/projects/17a95ff6870d40c6a94ee5243c78629d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:40.159735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:40.177228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5125ae4-46b9-4691-a32d-9da6362c297b service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:40.247848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:40.248503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:40.248783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:40.249092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] PATH_INFO: `/v3/users/9895bdfb5b2c4871a137574121d4774f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:40.250485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9895bdfb5b2c4871a137574121d4774f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:40.252880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:40.326328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:40.340818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9895bdfb5b2c4871a137574121d4774f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.358520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9895bdfb5b2c4871a137574121d4774f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.359395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9895bdfb5b2c4871a137574121d4774f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.361490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9895bdfb5b2c4871a137574121d4774f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.375901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9895bdfb5b2c4871a137574121d4774f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.404951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1108/2183] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:40 2022] DELETE /identity/v3/users/9895bdfb5b2c4871a137574121d4774f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:40.562424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:40.563968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:40.650041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5dc17-1141-416e-8204-853aa8333690 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:40.671402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:40.730109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:40.731056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:40.731550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:40.732029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:40.735253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:40.768536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:40.770367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:40.846175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:40.848477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63aa40-5b67-4b29-83bc-5ad7491b4012 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:40.865420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1076/2184] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:40.869555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fffc15-720b-4a6c-9c77-7e69b08b209a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:40.875325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:40.875988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:40.876261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:40.876528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] PATH_INFO: `/v3/projects/06c057000a584260921baed6fa37ebe1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:40.877765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c057000a584260921baed6fa37ebe1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:40.879772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:40.937323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c057000a584260921baed6fa37ebe1', 'request_id': 'req-8f258446-c34e-48cf-b23a-51c8670ac887'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.946475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c057000a584260921baed6fa37ebe1', 'request_id': 'req-8f258446-c34e-48cf-b23a-51c8670ac887'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.955111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c057000a584260921baed6fa37ebe1', 'request_id': 'req-8f258446-c34e-48cf-b23a-51c8670ac887'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.967402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] The token cache is being invalidate because project 06c057000a584260921baed6fa37ebe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:40.967693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.968475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8f258446-c34e-48cf-b23a-51c8670ac887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:40.970306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1109/2185] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:40 2022] DELETE /identity/v3/projects/06c057000a584260921baed6fa37ebe1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:40.977192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a689-91c7-4806-b201-446301f612e0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:40.977726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a689-91c7-4806-b201-446301f612e0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:40.978121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a689-91c7-4806-b201-446301f612e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.009444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f567a689-91c7-4806-b201-446301f612e0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:41.010997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f567a689-91c7-4806-b201-446301f612e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:41.096918 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1077/2186] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:41.098759 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.099036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.099305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.120738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:41.122017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:41.131202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1110/2187] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:41.137408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a689-91c7-4806-b201-446301f612e0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.156932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a689-91c7-4806-b201-446301f612e0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.157854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.158618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.158937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.159415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.160400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:41.160789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.162626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.182030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1078/2188] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:41.183766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.201578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe39460-5907-4789-89ca-bb22c41d6fc1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.209694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.210896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.211372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.211838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.212972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.216584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.263895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1111/2189] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:41 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:41.268017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.278076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d09a6d-5c4a-4b41-8592-291fea084763 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.338299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.338921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.339195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.339456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.340087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.342130 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.351843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:41.369174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1079/2190] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:41 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:41.370531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.381386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d549ef4-fb51-400f-9289-dcb15412d959 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.386797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.387558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.387826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.388121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.388735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.390362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.401435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1112/2191] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:41.405741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.426479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e800c4-0cf9-4d6b-8d2f-7da92e14139b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.432420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.433063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.433341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.433610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.434414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.436114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28415d-419d-482c-b2cb-bb85d6351a14 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.447510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1080/2192] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:41.451955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.462393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8702b98-8716-4e80-81f4-5c7d0e781b89 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.467318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.467981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.468258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.468534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] PATH_INFO: `/v3/projects/adf03eb999084cc797ba95d0b2a9be4b/users/77655edc605b4006a697914d8e74d794/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.471941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:41.487077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf03eb999084cc797ba95d0b2a9be4b, user_id=77655edc605b4006a697914d8e74d794, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.489837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.504302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1113/2193] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:41 2022] PUT /identity/v3/projects/adf03eb999084cc797ba95d0b2a9be4b/users/77655edc605b4006a697914d8e74d794/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:41.514741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.515027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.515292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.545052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] MFA Rules not processed for user `77655edc605b4006a697914d8e74d794`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:41.546278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:41.612327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1081/2194] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:41.623823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.626036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:41.691153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b099b-d659-4c85-8be5-0ad820791c00 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.695316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.695935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.696211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.696484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.705262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.706686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79072ef5-c00b-4c40-97ed-025f0bcc205e service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.715735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1114/2195] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:41.738112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:41.810534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8c91f-b5e9-4bef-a6eb-cb0dae1778ad None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:41.818791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:41.819804 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.820305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.820877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] PATH_INFO: `/v3/users/77655edc605b4006a697914d8e74d794` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.822730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77655edc605b4006a697914d8e74d794)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:41.824742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:41.878369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77655edc605b4006a697914d8e74d794'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:41.891024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77655edc605b4006a697914d8e74d794'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:41.891738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77655edc605b4006a697914d8e74d794'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:41.892362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77655edc605b4006a697914d8e74d794'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:41.905998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77655edc605b4006a697914d8e74d794'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:41.907380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:41.925727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1082/2196] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:41 2022] DELETE /identity/v3/users/77655edc605b4006a697914d8e74d794 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:41.927690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:41.928157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:41.928255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:41.948803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:41.950069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:42.013077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1115/2197] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:42.023199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.024810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:42.084104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc41224-7ab6-4a22-8d2e-e451e4ca884f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.087917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.088528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.088800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.089101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.098574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.100012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54776892-2f89-4c79-948e-e7df94bf7e76 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.109908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1083/2198] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:42.395399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.399461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:42.422843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78051ae-333d-4159-9368-064d2eb24221 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.428453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.429283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.429539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.429680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] PATH_INFO: `/v3/projects/adf03eb999084cc797ba95d0b2a9be4b` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.430921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf03eb999084cc797ba95d0b2a9be4b)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.432904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.482462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf03eb999084cc797ba95d0b2a9be4b', 'request_id': 'req-e22cf2fb-3df7-4b88-90d5-32416ec612a4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:42.488827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf03eb999084cc797ba95d0b2a9be4b', 'request_id': 'req-e22cf2fb-3df7-4b88-90d5-32416ec612a4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:42.497274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf03eb999084cc797ba95d0b2a9be4b', 'request_id': 'req-e22cf2fb-3df7-4b88-90d5-32416ec612a4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:42.508022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] The token cache is being invalidate because project adf03eb999084cc797ba95d0b2a9be4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:42.508309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:42.508969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-e22cf2fb-3df7-4b88-90d5-32416ec612a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:42.510511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1116/2199] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:42 2022] DELETE /identity/v3/projects/adf03eb999084cc797ba95d0b2a9be4b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:42.516632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.516952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.517230 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.538871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:42.540114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:42.603688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1084/2200] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:42.611989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.612244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.612517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.633318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:42.634529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:42.643439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1117/2201] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:42.647989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.658813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75270a8e-9d9c-47d2-8319-42c3285a8ae3 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.664124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.664740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.665052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.665318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.665979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:42.666264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.667865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.680534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1085/2202] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:42.685482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.692107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:42.696683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed14850-fb6f-4589-951d-3e9171fe0cc3 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.704263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.705190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.705494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.705768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.706452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.708868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.739858 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1118/2203] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:42 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:42.744153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.754946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829d059-e48e-49df-9883-749a1424d289 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.813245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.813888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.814157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.814431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.815103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.817070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.842903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1086/2204] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:42 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:42.844147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.855055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b18ac-9fae-4b67-8043-07d8459ed80c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.860130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.860738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.861052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.861320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.861969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.863578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.874220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1119/2205] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:42.878477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.888938 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ed511-1101-45a2-9c7d-fca7e7889625 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.894812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.895412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.895698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.895967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.896608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.898273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6d7c1-dc50-457f-85c5-03f614bb1e92 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.900575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:42.911194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1087/2206] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:42.915891 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:42.926121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7dcbd-a238-4137-a890-9fd1e969f37c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:42.930980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:42.931641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.931921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.932195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] PATH_INFO: `/v3/projects/62ad20021ed0483186dad6591ba5ce67/users/299a8c2de0cf4f0aa1570c718fe04e52/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:42.948379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ad20021ed0483186dad6591ba5ce67, user_id=299a8c2de0cf4f0aa1570c718fe04e52, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:42.951101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:42.965012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1120/2207] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:42 2022] PUT /identity/v3/projects/62ad20021ed0483186dad6591ba5ce67/users/299a8c2de0cf4f0aa1570c718fe04e52/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:42.976092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba695-6610-4867-9492-99522034303b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:42.977075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba695-6610-4867-9492-99522034303b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:42.977075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dba695-6610-4867-9492-99522034303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:43.007773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f7dba695-6610-4867-9492-99522034303b None None] MFA Rules not processed for user `299a8c2de0cf4f0aa1570c718fe04e52`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:43.009039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f7dba695-6610-4867-9492-99522034303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:43.089580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1088/2208] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:43.100754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:43.102552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:43.164874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410da165-34a3-4e0d-ac35-1fad46218c71 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:43.169356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:43.169983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:43.170265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:43.170538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:43.179602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:43.181091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:43.190727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1121/2209] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:43.539645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dba695-6610-4867-9492-99522034303b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:43.550851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dba695-6610-4867-9492-99522034303b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:43.554807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:43.555467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:43.555746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:43.556059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:43.560102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:43.634759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:43.636128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0c9d75-4871-43e6-a1b4-593aa89ac171 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:43.647085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1089/2210] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:51.032468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:51.034150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:51.046960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61885943-7b32-4c17-b4d6-855a54c07080 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:51.051468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:51.052072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:51.052347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:51.052614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.054468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:51.119662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:51.121146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:51.131496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1122/2211] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:52.009262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4b8d-cc01-4920-8e7a-13669209d6ac None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.009774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4b8d-cc01-4920-8e7a-13669209d6ac None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.009866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd4b8d-cc01-4920-8e7a-13669209d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.029374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dffd4b8d-cc01-4920-8e7a-13669209d6ac None None] MFA Rules not processed for user `299a8c2de0cf4f0aa1570c718fe04e52`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:52.030506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dffd4b8d-cc01-4920-8e7a-13669209d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.039071 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1090/2212] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:52.563526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.565256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.656079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:52.684549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82368285-ab73-441a-a525-b72a276b1f64 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:52.690638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:52.691492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.691838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.692179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] PATH_INFO: `/v3/users/8c0549313b8f43098629b2a8b1451a4f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.693952 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0549313b8f43098629b2a8b1451a4f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:52.697884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:52.758517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0549313b8f43098629b2a8b1451a4f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.768205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0549313b8f43098629b2a8b1451a4f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.769348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0549313b8f43098629b2a8b1451a4f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.770325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0549313b8f43098629b2a8b1451a4f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.780053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0549313b8f43098629b2a8b1451a4f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:52.801566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1123/2213] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:52 2022] DELETE /identity/v3/users/8c0549313b8f43098629b2a8b1451a4f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:52.807132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:52.807604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:52.807903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:52.831898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:52.833409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:52.908614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1091/2214] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:52.918921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:52.920361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.001305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6e4512-811b-45df-86f1-491de8f59286 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.006875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.007757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.008294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.008728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.024875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.026619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0179e98-7bba-4bf9-bc2a-f1eebf5a7368 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.039304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1124/2215] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:53.350171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.351632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.364256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1690b0-dbdb-4d87-98d1-16edbe11d796 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.369050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.369827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.370202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.370535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] PATH_INFO: `/v3/projects/eca4e96a9fdd4b25ae6aee54432f4e59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.371727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=eca4e96a9fdd4b25ae6aee54432f4e59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.373751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.385458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:53.422796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eca4e96a9fdd4b25ae6aee54432f4e59', 'request_id': 'req-623aec38-72a0-42e2-b72c-354eb97d1aee'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:53.429448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eca4e96a9fdd4b25ae6aee54432f4e59', 'request_id': 'req-623aec38-72a0-42e2-b72c-354eb97d1aee'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:53.437763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eca4e96a9fdd4b25ae6aee54432f4e59', 'request_id': 'req-623aec38-72a0-42e2-b72c-354eb97d1aee'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:53.448807 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] The token cache is being invalidate because project eca4e96a9fdd4b25ae6aee54432f4e59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:53.449120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:53.449763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-623aec38-72a0-42e2-b72c-354eb97d1aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:53.451268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1092/2216] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:53 2022] DELETE /identity/v3/projects/eca4e96a9fdd4b25ae6aee54432f4e59 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:53.456431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.456715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.457086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.476943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:53.478212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.552327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1125/2217] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:53.560617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.560933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.561248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.581231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:53.582472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:53.590576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1093/2218] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:53.595071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.605449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47190e8-cca2-4da4-9c40-c2efbd7aa0f6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.610116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.610740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.611035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.611310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.611974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:53.612273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.613982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.615666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:53.632191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1126/2219] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:53.633512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.643945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec85c6da-256b-4a5d-9813-4b26a3039b0b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.649475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.650158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.650659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.650781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.651444 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.653540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.684762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1094/2220] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:53.689428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.699816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a30583-6c7f-44b1-95f4-bcb14e804600 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.763422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.764122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.764400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.764665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.765552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.768340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.797259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1127/2221] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:53.803097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.816667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5c63c-7cd9-48be-8ac0-d112483634a1 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.822817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.823501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.823917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.824315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.825104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.826732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.837597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1095/2222] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:53.841862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.856549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa4be9b-ca56-4d89-b49d-92644a3b409e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.864221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.865210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.865653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.866041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.866733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.869252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1056154-3fb2-48ed-be2a-f21cb8a18c1d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.887431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1128/2223] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:53.892305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:53.908525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bdaa4-7891-4b88-a078-5bf7c56155ac admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:53.914126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:53.914852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.915172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.915587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] PATH_INFO: `/v3/projects/a59154ca6256460393236bb285cb135b/users/908d7c6a04ac4bc3ba08f64ae8e056fe/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:53.931916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59154ca6256460393236bb285cb135b, user_id=908d7c6a04ac4bc3ba08f64ae8e056fe, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:53.934750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:53.956742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1096/2224] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:53 2022] PUT /identity/v3/projects/a59154ca6256460393236bb285cb135b/users/908d7c6a04ac4bc3ba08f64ae8e056fe/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:53.974248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:53.974751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:53.975104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:54.004532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] MFA Rules not processed for user `908d7c6a04ac4bc3ba08f64ae8e056fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:54.005764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:54.074854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1129/2225] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:54.086698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:54.088664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:54.150493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02dab9-8915-40ee-9233-f9e1808b6f82 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:54.154478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:54.155083 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:54.155360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:54.155640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:54.165177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:54.166637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c337f4-c9c5-4404-afa9-290a55906a25 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:54.176150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1097/2226] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:04.071527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:04.171756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:11.465218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.470701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:11.577699 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcb4aa-9f54-4f75-9918-ed32db6a839b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.582867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.583525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.583810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.584083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] PATH_INFO: `/v3/users/903147b749ff4fc88e8e75b305c05efd` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.585286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=903147b749ff4fc88e8e75b305c05efd)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.587243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.652802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903147b749ff4fc88e8e75b305c05efd'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:11.661883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903147b749ff4fc88e8e75b305c05efd'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:11.662917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903147b749ff4fc88e8e75b305c05efd'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:11.663648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903147b749ff4fc88e8e75b305c05efd'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:11.670989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903147b749ff4fc88e8e75b305c05efd'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:11.688472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1130/2227] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:11 2022] DELETE /identity/v3/users/903147b749ff4fc88e8e75b305c05efd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:11.693919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.694420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.694509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.715159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:11.716646 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:11.784261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1098/2228] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:11.795244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:11.796716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:11.860375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f440750-145b-42c5-bd09-a9fe45784ab8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:11.864307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:11.864969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:11.865248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:11.865516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:11.876671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:11.878208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:11.888310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1131/2229] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:12.217148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:12.218812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:12.234760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb3def-785d-4912-bb73-69a47c7d3f82 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:12.241010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:12.241667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:12.241943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:12.242292 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] PATH_INFO: `/v3/projects/31e01eff8f1446a18497195dabfe58bf` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:12.243407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e01eff8f1446a18497195dabfe58bf)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:12.245399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:12.301296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e01eff8f1446a18497195dabfe58bf', 'request_id': 'req-fa51f318-259a-40c3-b433-b0c932c899a3'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.307716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e01eff8f1446a18497195dabfe58bf', 'request_id': 'req-fa51f318-259a-40c3-b433-b0c932c899a3'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.314794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e01eff8f1446a18497195dabfe58bf', 'request_id': 'req-fa51f318-259a-40c3-b433-b0c932c899a3'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.325508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] The token cache is being invalidate because project 31e01eff8f1446a18497195dabfe58bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:12.325796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.326433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.327948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1099/2230] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:12 2022] DELETE /identity/v3/projects/31e01eff8f1446a18497195dabfe58bf => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:12.332077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:12.342047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb701b4-a3a2-4a0d-9185-ca4293c78a8a service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:12.360776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:12.402536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:12.403176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:12.403456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:12.403723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] PATH_INFO: `/v3/users/4c4c23fe08f44db59d9fd1cb74701ece` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:12.404906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c4c23fe08f44db59d9fd1cb74701ece)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:12.406832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:12.453994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c4c23fe08f44db59d9fd1cb74701ece'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.461494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c4c23fe08f44db59d9fd1cb74701ece'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.462133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c4c23fe08f44db59d9fd1cb74701ece'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.462758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c4c23fe08f44db59d9fd1cb74701ece'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.469733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c4c23fe08f44db59d9fd1cb74701ece'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.487705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1132/2231] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:12 2022] DELETE /identity/v3/users/4c4c23fe08f44db59d9fd1cb74701ece => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:12.778219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:12.780723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:12.809575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51f318-259a-40c3-b433-b0c932c899a3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:12.868727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:12.869315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:12.869594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:12.869866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] PATH_INFO: `/v3/projects/eeef3b7cf98a4e318f5e35b3a258cffa` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:12.870984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeef3b7cf98a4e318f5e35b3a258cffa)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:12.872939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:12.904965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:12.923801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeef3b7cf98a4e318f5e35b3a258cffa', 'request_id': 'req-5b029940-6373-4d11-ba8b-25113fcde1d8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.931002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeef3b7cf98a4e318f5e35b3a258cffa', 'request_id': 'req-5b029940-6373-4d11-ba8b-25113fcde1d8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.941899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeef3b7cf98a4e318f5e35b3a258cffa', 'request_id': 'req-5b029940-6373-4d11-ba8b-25113fcde1d8'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.953890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] The token cache is being invalidate because project eeef3b7cf98a4e318f5e35b3a258cffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:12.954173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.954824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:12.956350 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1100/2232] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:12 2022] DELETE /identity/v3/projects/eeef3b7cf98a4e318f5e35b3a258cffa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:12.960619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:12.962278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:12.976421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9918ec-cdfe-4d22-8dec-2f7fa2dd1309 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.040192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.040843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.041148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.041414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] PATH_INFO: `/v3/users/2a535dd0cfce43fcb4d2b616b3e28eb1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.042551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a535dd0cfce43fcb4d2b616b3e28eb1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.044493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.096437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:13.099830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a535dd0cfce43fcb4d2b616b3e28eb1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.110781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a535dd0cfce43fcb4d2b616b3e28eb1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.112400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a535dd0cfce43fcb4d2b616b3e28eb1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.113303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a535dd0cfce43fcb4d2b616b3e28eb1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.121621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0cb04b5b-68bb-4c7f-8dfe-2c1695d0b1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a535dd0cfce43fcb4d2b616b3e28eb1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.140629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1133/2233] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:12 2022] DELETE /identity/v3/users/2a535dd0cfce43fcb4d2b616b3e28eb1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:13.434034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.435506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.449520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b029940-6373-4d11-ba8b-25113fcde1d8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.513139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.513962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.514307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.514679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] PATH_INFO: `/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.516332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e38fb52e9fb44a7b2d77788c2e88b83)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.518299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.581708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e38fb52e9fb44a7b2d77788c2e88b83', 'request_id': 'req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.588849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e38fb52e9fb44a7b2d77788c2e88b83', 'request_id': 'req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.598002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e38fb52e9fb44a7b2d77788c2e88b83', 'request_id': 'req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.608465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] The token cache is being invalidate because project 9e38fb52e9fb44a7b2d77788c2e88b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:13.608750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.609428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ee0c0614-b2dd-49fc-bb07-5b2d828c1b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:13.610925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1101/2234] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:13 2022] DELETE /identity/v3/projects/9e38fb52e9fb44a7b2d77788c2e88b83 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:13.616377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.616871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.617025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.638284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:13.639451 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.707287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1134/2235] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:13.715720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.716074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.716437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.737865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:13.739092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:13.747895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1102/2236] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:13.752390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.763624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f7236-d605-4b70-90fa-980b2ac8d757 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.769428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.770102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.770364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.770626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.771311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:13.771626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.773280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.786085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1135/2237] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:13.791288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.801121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379b1cd-6073-484d-9adc-1d1fc97bd8b0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.809034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.810132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.810550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.810926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.811853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.814779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.850792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1103/2238] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:13 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:13.856063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.867970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fda87-f8e3-40bc-8535-ef75460c6622 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.933111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.933659 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.933933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.934201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.934857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.936854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.961204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1136/2239] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:13 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:13.965319 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:13.977847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be842fb-a66f-44fb-8775-46f62097a701 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:13.983233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:13.984009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:13.984293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:13.984562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:13.985256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:13.987028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:13.998911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1104/2240] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:14.002865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:14.013122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6f58-100d-40ca-9f98-6ab7c1cfb12d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:14.017869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:14.018498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:14.018835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:14.019097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:14.019760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:14.027587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd5fb6-c4a2-41cb-84c9-eb176edc68df admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:14.040668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1137/2241] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:14.045443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:14.052457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:14.056857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02dd55b-a852-4aaa-827b-7842a68b1416 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:14.063715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:14.064486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:14.065003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:14.065468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] PATH_INFO: `/v3/projects/150b5750dfc94e3fb8a8ce7032a9db3d/users/ed54cffa463e4d209ceccc48ab1858ad/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:14.083280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=150b5750dfc94e3fb8a8ce7032a9db3d, user_id=ed54cffa463e4d209ceccc48ab1858ad, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:14.086022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:14.102817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1105/2242] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:14 2022] PUT /identity/v3/projects/150b5750dfc94e3fb8a8ce7032a9db3d/users/ed54cffa463e4d209ceccc48ab1858ad/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:14.176426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:14.176705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:14.177007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:14.208185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] MFA Rules not processed for user `ed54cffa463e4d209ceccc48ab1858ad`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:14.209374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:14.225439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:14.289875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1138/2243] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:14.301245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:14.304991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:14.367795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6bef90-3ca1-4ed2-9f2c-0131102ba149 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:14.371583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:14.372221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:14.372501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:14.372768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:14.382733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:14.385348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00935a7-117f-483d-8352-64251b608f5d service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:14.395718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1106/2244] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:24.285920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=20 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:24.391170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:25.178648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:25.181184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:25.258417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd16fd-e320-4377-9ad3-f0862550bc76 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:25.269777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:25.270631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:25.270829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:25.271103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] PATH_INFO: `/v3/users/b1dab0ff34a34cd9af03a2d5edf5dcb6` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:25.272397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1dab0ff34a34cd9af03a2d5edf5dcb6)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:25.275223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:25.370199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1dab0ff34a34cd9af03a2d5edf5dcb6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:25.382003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1dab0ff34a34cd9af03a2d5edf5dcb6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:25.384199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1dab0ff34a34cd9af03a2d5edf5dcb6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:25.385102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1dab0ff34a34cd9af03a2d5edf5dcb6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:25.400268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1dab0ff34a34cd9af03a2d5edf5dcb6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:25.423435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1139/2245] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:25 2022] DELETE /identity/v3/users/b1dab0ff34a34cd9af03a2d5edf5dcb6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:25.429371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:25.429935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:25.429935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:25.450454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:25.451967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:25.527495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1107/2246] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:25.557126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:25.557126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:25.665306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255ea96-b583-4bae-a16c-1ca71937c300 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:25.680478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:25.681812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:25.681812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:25.681812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:25.694719 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:25.696440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af5ad0-743f-4603-aa58-9dbb84705b6d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:25.708810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1140/2247] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:26.131283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.132840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.159479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b1b53-0db2-4b12-8edb-c0d03c8b60c9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.166389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.167565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.167833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.168123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] PATH_INFO: `/v3/projects/1db953a629a944a8b73a3721b461dc94` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.169367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1db953a629a944a8b73a3721b461dc94)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.171305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.232990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1db953a629a944a8b73a3721b461dc94', 'request_id': 'req-d586fb89-f828-4d08-b430-5928735339b1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:26.240204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1db953a629a944a8b73a3721b461dc94', 'request_id': 'req-d586fb89-f828-4d08-b430-5928735339b1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:26.251027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1db953a629a944a8b73a3721b461dc94', 'request_id': 'req-d586fb89-f828-4d08-b430-5928735339b1'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:26.262789 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] The token cache is being invalidate because project 1db953a629a944a8b73a3721b461dc94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:26.263157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:26.264018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d586fb89-f828-4d08-b430-5928735339b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:26.265887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1108/2248] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:26 2022] DELETE /identity/v3/projects/1db953a629a944a8b73a3721b461dc94 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:26.271546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.272128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.272539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.297477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:26.299346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.329214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:26.377572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1141/2249] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:26.387861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.388146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.388440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.411038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:26.412630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.422796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1109/2250] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:26.428255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.442910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d55e0-d04d-4d8f-98cc-165065da41e4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.448927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.450006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.450367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.450804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.451865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:26.452353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.455213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.474173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1142/2251] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:52:26.481912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.493355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0f3fb-c4c1-4f59-8f76-633b2a0d0a88 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.503247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.504213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.504469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.504738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.505423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.507468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.541937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1110/2252] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:26 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:26.547633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.559242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784cf9e-a177-4eed-b1ee-f0cc4d0342be admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.635557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.636497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.636790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.637147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.637859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.639818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.671710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1143/2253] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:26 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:26.675922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.688328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352848f8-121f-4c57-8741-7d0258442f64 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.693782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.694618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.694967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.695236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.696226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.698317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.720744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1111/2254] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:52:26.724871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.736239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82b7f2-cc97-4155-9926-dc3efc357260 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.741994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.742865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.743310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.743596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.745045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.749514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5eb310-138c-4eda-982c-f5ff4b6dd3a2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.762013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1144/2255] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:26.766581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.777498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60abd8a1-1159-4979-9949-ce08f59552a0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:26.794058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:26.794783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.795068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.795341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] PATH_INFO: `/v3/projects/dbb030840abe441a964c521dc69f230e/users/54485e238883496a9b27fe380d757b7a/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.814618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb030840abe441a964c521dc69f230e, user_id=54485e238883496a9b27fe380d757b7a, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:26.817483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:26.835167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1112/2256] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:26 2022] PUT /identity/v3/projects/dbb030840abe441a964c521dc69f230e/users/54485e238883496a9b27fe380d757b7a/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:26.845817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:26.846125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:26.846357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:26.879184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] MFA Rules not processed for user `54485e238883496a9b27fe380d757b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:26.880465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:26.959618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1145/2257] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:26.971837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:26.973985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:27.000595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:27.057434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c67442a-e9e6-454c-88ed-d755d8708ad3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:27.061811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:27.062480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:27.062760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:27.063042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:27.073582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:27.075119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d71551-1b95-46a1-8cc5-e7a9261a8628 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:27.085922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1113/2258] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:27.314832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:27.316943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:27.341199 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:27.408615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3af45e-7ac1-403b-a050-70406a571595 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:27.414800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:27.415936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:27.416339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:27.416740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] PATH_INFO: `/v3/users/908d7c6a04ac4bc3ba08f64ae8e056fe` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:27.418102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908d7c6a04ac4bc3ba08f64ae8e056fe)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:27.420665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:27.505899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908d7c6a04ac4bc3ba08f64ae8e056fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:27.521396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908d7c6a04ac4bc3ba08f64ae8e056fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:27.522042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908d7c6a04ac4bc3ba08f64ae8e056fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:27.522632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908d7c6a04ac4bc3ba08f64ae8e056fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:27.531287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908d7c6a04ac4bc3ba08f64ae8e056fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:27.550651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1146/2259] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:27 2022] DELETE /identity/v3/users/908d7c6a04ac4bc3ba08f64ae8e056fe => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:27.555248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:27.555751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:27.555751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:27.576076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:27.577346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:27.642339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1114/2260] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:27.652629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:27.654168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:27.717165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb158c4-a71d-4319-9733-b5c9901b9b56 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:27.721548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:27.722209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:27.722479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:27.722744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:27.733375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:27.734818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5368a9e-1ff1-4116-b932-654b606920f6 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:27.745053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1147/2261] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:28.053333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.054885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.067264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713f5bf-2b73-4a4d-99e1-a6d337d18f01 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.073940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.074898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.075371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.075607 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] PATH_INFO: `/v3/projects/a59154ca6256460393236bb285cb135b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.077256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59154ca6256460393236bb285cb135b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.079584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.134294 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59154ca6256460393236bb285cb135b', 'request_id': 'req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.141211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59154ca6256460393236bb285cb135b', 'request_id': 'req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.149495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59154ca6256460393236bb285cb135b', 'request_id': 'req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.162365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] The token cache is being invalidate because project a59154ca6256460393236bb285cb135b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:28.162649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.163587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-dfe5aa58-e9fe-4630-8367-43e13eef3b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.165332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1115/2262] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:28 2022] DELETE /identity/v3/projects/a59154ca6256460393236bb285cb135b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:28.170668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.170971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.171283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.191877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:28.193244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.229237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:28.272305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1148/2263] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:28.275193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.275708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.276162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.303762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:28.305287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.314946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1116/2264] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:28.319782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.331847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a869fb7-98e7-4181-b35e-9cbcbd30a944 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.338182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.339029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.339743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.339921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.340681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:28.341128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.342959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.358041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1149/2265] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:28.366679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.377160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b16488a-88d1-4234-b6c3-4d9b4ac209a5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.382183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.382805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.383089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.383513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.384018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.386633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.420609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1117/2266] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:28.422078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.433708 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41be5c-e479-478f-b4b4-7586a2c8c2e2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.496473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.497129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.497407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.497675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.498346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.500273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.533855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.537995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.538886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1150/2267] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:28.542959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.554563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68247726-e22c-4a1d-81b6-df5e3d5f7f75 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.554775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19b49-c114-47de-bfdd-f86f618fa1ab admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.562855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.564416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.564700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.564996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] PATH_INFO: `/v3/users/fca0d685cb9a4603a85a0c5dd5eb8ab9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.566228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.566885 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.567163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.567431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.568086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.569117 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca0d685cb9a4603a85a0c5dd5eb8ab9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.571167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.571305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.588328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1151/2268] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:28.589454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.591627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:28.601391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890a935-904e-4d6b-8b7b-ef0be5742a30 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.612228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.613449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.613449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.613727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.614424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.616457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.629883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1152/2269] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:28.636081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.650604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca0d685cb9a4603a85a0c5dd5eb8ab9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.651049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f39a42-db17-4a02-9035-abe1ec9306a1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.658890 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.660506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.660968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.661612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] PATH_INFO: `/v3/projects/fa70232219304082b5ee608a7ea61c22/users/11c76c57183e406c87f434841fc13b43/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.669535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca0d685cb9a4603a85a0c5dd5eb8ab9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.670383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca0d685cb9a4603a85a0c5dd5eb8ab9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.671168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca0d685cb9a4603a85a0c5dd5eb8ab9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.695410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa70232219304082b5ee608a7ea61c22, user_id=11c76c57183e406c87f434841fc13b43, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.698383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffd8e4-5bcb-4d25-b319-9a8f49a4cb71 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:28.699801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2be9b0cb-0496-4c63-8d36-32b560508509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca0d685cb9a4603a85a0c5dd5eb8ab9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:28.717649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1118/2270] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:28 2022] DELETE /identity/v3/users/fca0d685cb9a4603a85a0c5dd5eb8ab9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:28.723023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1153/2271] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:28 2022] PUT /identity/v3/projects/fa70232219304082b5ee608a7ea61c22/users/11c76c57183e406c87f434841fc13b43/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:28.724694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.725036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.725223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.725374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.725788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.726269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.747184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:28.747184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.761408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:28.763401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.846748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1154/2272] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:28.855299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.857097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.879010 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:28.895169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1119/2273] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:28.905554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:28.909839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:28.960989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd2d9d-421a-448e-bedb-dc93e70e3120 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:28.966763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:28.967776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:28.968189 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:28.968615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:28.986185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:28.988335 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.000868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b5612-f69f-4183-8b56-09cf0db01339 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:29.003793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1155/2274] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:52:29.007527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:29.008779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.009329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.010602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.040387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:29.042377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.072738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1120/2275] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:29.481088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:29.483525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.502600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3e2d-f0ee-41be-8ca3-1695e8720162 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:29.507946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:29.508640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.508944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.509221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] PATH_INFO: `/v3/projects/8e89402e388d47e7ba42747b2d4c10f2` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.510433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e89402e388d47e7ba42747b2d4c10f2)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:29.512410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.579732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e89402e388d47e7ba42747b2d4c10f2', 'request_id': 'req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.587141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e89402e388d47e7ba42747b2d4c10f2', 'request_id': 'req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.600285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e89402e388d47e7ba42747b2d4c10f2', 'request_id': 'req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.621782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] The token cache is being invalidate because project 8e89402e388d47e7ba42747b2d4c10f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:29.621995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.622679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ae67ad86-5f6a-481c-bea7-aa80ca8f9fe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.624336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1156/2276] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:29 2022] DELETE /identity/v3/projects/8e89402e388d47e7ba42747b2d4c10f2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:29.628639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:29.650800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf07975-9739-40bf-826f-9622a266d7c5 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:29.696320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:29.731863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:29.732231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.732374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.732558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] PATH_INFO: `/v3/users/cdfa32e6ec8c49e589154e665ea6e70f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.733734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdfa32e6ec8c49e589154e665ea6e70f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:29.735683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.796677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdfa32e6ec8c49e589154e665ea6e70f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.812753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdfa32e6ec8c49e589154e665ea6e70f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.814135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdfa32e6ec8c49e589154e665ea6e70f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.814922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdfa32e6ec8c49e589154e665ea6e70f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.835708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdfa32e6ec8c49e589154e665ea6e70f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:29.854293 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1121/2277] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:29 2022] DELETE /identity/v3/users/cdfa32e6ec8c49e589154e665ea6e70f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:30.328297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.328297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.328297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.330499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:30.333246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.396379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] MFA Rules not processed for user `11c76c57183e406c87f434841fc13b43`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:30.399275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.479395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e2008-9e25-4b73-b0f6-2a9bbc02d8e4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:30.485114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:30.486006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.486299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.486632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] PATH_INFO: `/v3/projects/c1ed373f55ca49939515fa87603eaa70` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.488682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ed373f55ca49939515fa87603eaa70)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:30.490722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:30.508067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1157/2278] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:30.517265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:30.518920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.571031 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ed373f55ca49939515fa87603eaa70', 'request_id': 'req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.582543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ed373f55ca49939515fa87603eaa70', 'request_id': 'req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.586722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:30.593767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:30.610007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ed373f55ca49939515fa87603eaa70', 'request_id': 'req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.627749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] The token cache is being invalidate because project c1ed373f55ca49939515fa87603eaa70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:30.628272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.629737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5cc1f15d-9fbf-4125-98de-c483c5bbe18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.631951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1122/2279] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:30 2022] DELETE /identity/v3/projects/c1ed373f55ca49939515fa87603eaa70 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:30.637722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.638006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.638277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.645569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a428a-5851-4164-a039-c6c4c4f8216e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:30.662967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:30.664203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.701320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:30.702004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.702276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.702577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.704344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.728528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1123/2280] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:30.729286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:30.735899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.748462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efe63c1-f931-4fe4-a9a8-1b6e81b80920 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:30.753469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:30.754109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.754386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.754657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] PATH_INFO: `/v3/users/ed54cffa463e4d209ceccc48ab1858ad` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.755792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed54cffa463e4d209ceccc48ab1858ad)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:30.757776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:30.765076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:30.766532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e73cc8-bffb-496d-ac76-a27bf5617a25 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:30.776460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1158/2281] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:30.777829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.778119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.778385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.804838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:30.806084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.814109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed54cffa463e4d209ceccc48ab1858ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.819899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1159/2282] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:30.825101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:30.825705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed54cffa463e4d209ceccc48ab1858ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.826461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed54cffa463e4d209ceccc48ab1858ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.827385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed54cffa463e4d209ceccc48ab1858ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.835725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d084a5-ecd5-4d00-bef4-df3b7c18de2d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:30.841289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:30.841907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.842194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.842533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.843254 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:30.843550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:30.844876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9dc24b9b-922b-4e44-98be-e06c04bb6a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed54cffa463e4d209ceccc48ab1858ad'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:30.845468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:30.862689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1124/2283] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:30 2022] DELETE /identity/v3/users/ed54cffa463e4d209ceccc48ab1858ad => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:30.866595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1160/2284] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:52:30.867623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18514208-830c-4ada-bca0-7441e827502f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.867778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:30.867933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18514208-830c-4ada-bca0-7441e827502f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.868181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18514208-830c-4ada-bca0-7441e827502f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.870333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.885527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803514cb-13ef-472b-b4d5-53c8abda4122 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:30.898223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-18514208-830c-4ada-bca0-7441e827502f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:30.900036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-18514208-830c-4ada-bca0-7441e827502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.983948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:30.988177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:30.990110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.990979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.991833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.992867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:30.997339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.001984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1125/2285] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:31.014844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18514208-830c-4ada-bca0-7441e827502f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.016569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-18514208-830c-4ada-bca0-7441e827502f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:31.052237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1161/2286] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/projects => generated 407 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:31.054299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.060581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:31.097933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18514208-830c-4ada-bca0-7441e827502f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.148498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478069-aae4-4c98-83c4-0dee9d2943ac admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.153427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.154174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.154416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.154584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.155209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.157165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.168401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.169154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.169540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.169820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.181385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.183270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.183886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1162/2287] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:31 2022] POST /identity/v3/users => generated 304 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:31.188023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.196868 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1126/2288] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:31.205536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87940-0c1c-4a46-802c-57680bc238ec admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.212378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.213063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.213347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.213622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.214284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.216021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.227261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1163/2289] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:52:31.232669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.244058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd52357-29c1-43bd-a741-beddea766cb0 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.249825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.250536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.250830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.251274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.252122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.254207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f4e14-117b-4ef6-b790-e20362439a3c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.268823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1127/2290] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:31.273167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.287461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae341eac-f0c8-4108-8388-d17ad1d5cffb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.301011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.301720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.301998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.302265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] PATH_INFO: `/v3/projects/1a34d8d2a04046d1b16c778106a52f5e/users/7bde6bd467c34da9a8f27f528735d85f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.325252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a34d8d2a04046d1b16c778106a52f5e, user_id=7bde6bd467c34da9a8f27f528735d85f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.331516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.349576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1164/2291] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:31 2022] PUT /identity/v3/projects/1a34d8d2a04046d1b16c778106a52f5e/users/7bde6bd467c34da9a8f27f528735d85f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:31.359954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.360457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.360740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.400284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] MFA Rules not processed for user `7bde6bd467c34da9a8f27f528735d85f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:31.401638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:31.482032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1128/2292] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:31 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:31.486367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.538866 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.541217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:31.573906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f2f51-5c43-4b2e-9918-0698257e4c96 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.581130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.581130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.581130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.581130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.581492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.583827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.615412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:31.637260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1165/2293] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:31 2022] POST /identity/v3/projects => generated 409 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:31.640679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.685511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c375b-615b-4cfe-ad58-92bcd50931e6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.706799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:31.749625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04038f-33bc-4285-ba76-a7e56308f830 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.756735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.757448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.757939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.758305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.759142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.761622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.769539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.770416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.771578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.771578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] PATH_INFO: `/v3/projects/150b5750dfc94e3fb8a8ce7032a9db3d` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.773323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=150b5750dfc94e3fb8a8ce7032a9db3d)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.776069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.795527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1166/2294] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:31 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:31.797001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.816784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9f681-0bd7-459a-b260-20dcf6b67484 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.824434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.825106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.825427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.825873 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.826577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.829016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.843396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1167/2295] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:31.850029 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.860425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150b5750dfc94e3fb8a8ce7032a9db3d', 'request_id': 'req-60161a1b-8faf-485a-b379-fb42c1dbaf8b'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:31.862462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8b47e-76ab-43c1-9714-5eb8302bc527 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:31.869296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150b5750dfc94e3fb8a8ce7032a9db3d', 'request_id': 'req-60161a1b-8faf-485a-b379-fb42c1dbaf8b'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:31.880347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150b5750dfc94e3fb8a8ce7032a9db3d', 'request_id': 'req-60161a1b-8faf-485a-b379-fb42c1dbaf8b'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:31.894568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] The token cache is being invalidate because project 150b5750dfc94e3fb8a8ce7032a9db3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:31.894771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:31.895411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:31.897754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1129/2296] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:31 2022] DELETE /identity/v3/projects/150b5750dfc94e3fb8a8ce7032a9db3d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:31.947296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:31.947991 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:31.948269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:31.948547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] PATH_INFO: `/v3/projects/a2f48805059949248392aa1aa929fb43/users/c5183fbd21824f2995253f615369486f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:31.967086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f48805059949248392aa1aa929fb43, user_id=c5183fbd21824f2995253f615369486f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:31.970037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:31.987555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1168/2297] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:31 2022] PUT /identity/v3/projects/a2f48805059949248392aa1aa929fb43/users/c5183fbd21824f2995253f615369486f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:31.992191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:31.993751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.007556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60161a1b-8faf-485a-b379-fb42c1dbaf8b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:32.083262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:32.083896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.084175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.084446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.085154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:32.086887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8217c-755c-4715-a29c-101bb6f6d176 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:32.100027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1130/2298] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:32.104644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:32.116861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa905fb-c774-458c-9b26-08263aa3a29d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:32.121977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:32.122658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.122932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.123197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] PATH_INFO: `/v3/projects/a2f48805059949248392aa1aa929fb43/users/c5183fbd21824f2995253f615369486f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.125281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f48805059949248392aa1aa929fb43, user_id=c5183fbd21824f2995253f615369486f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:32.128544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4b0c5-6f2d-4823-9d37-7f918370b73a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:32.141053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1169/2299] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:32 2022] PUT /identity/v3/projects/a2f48805059949248392aa1aa929fb43/users/c5183fbd21824f2995253f615369486f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:32.152838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.153061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.153313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.182478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:32.218702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] MFA Rules not processed for user `c5183fbd21824f2995253f615369486f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:32.220099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.229275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.229547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.229825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.251466 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] MFA Rules not processed for user `11c76c57183e406c87f434841fc13b43`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:32.252764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.307278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1131/2300] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:32 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:32.320386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1170/2301] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:32.322541 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:32.324533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.397417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817213a-4da8-4121-9634-8eed653a3f47 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:32.401107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:32.401729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.402037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.402313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.412349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:32.413750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af51e6-a72a-42b2-ae80-028859809ed5 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:32.423565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1132/2302] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:32 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:32.592016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:32.612507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edc3da8-75db-4257-be03-dcbe439be6a4 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:32.617983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:32.618560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:32.618906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:32.619237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:32.622756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:32.674992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:32.705755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:32.707267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:32.717474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1171/2303] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:32 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:35.345722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cdc2-2ceb-4369-9363-dd28ff23c371 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:35.346366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cdc2-2ceb-4369-9363-dd28ff23c371 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:35.346366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d0cdc2-2ceb-4369-9363-dd28ff23c371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:35.368730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d8d0cdc2-2ceb-4369-9363-dd28ff23c371 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:35.370013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d8d0cdc2-2ceb-4369-9363-dd28ff23c371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:35.426729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1133/2304] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:52:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:35.469484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:35.471763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:35.568015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f07fa1c-4b1f-4a89-a4e5-d3dac6d5dadb service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:35.573685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:35.574299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:35.574567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:35.574853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:35.583583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:35.585176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:35.593760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1172/2305] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:52:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:52:36.200246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8237-b09e-4db0-ab03-5cf12c12d3e5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:36.200532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8237-b09e-4db0-ab03-5cf12c12d3e5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:36.200803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d8237-b09e-4db0-ab03-5cf12c12d3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:36.221027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-302d8237-b09e-4db0-ab03-5cf12c12d3e5 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:36.222310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-302d8237-b09e-4db0-ab03-5cf12c12d3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:36.229883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1134/2306] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:36.245491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:36.256807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bcbe6-4d06-43ff-99de-100ce79f7025 service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:36.262256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:36.262903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:36.263182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:36.263446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:36.272863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:36.274942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:36.283459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1173/2307] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:52:36.564704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b33-97d5-4ecb-9796-70ad0a310400 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:36.564704 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b33-97d5-4ecb-9796-70ad0a310400 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:36.565493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501b2b33-97d5-4ecb-9796-70ad0a310400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:36.586950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-501b2b33-97d5-4ecb-9796-70ad0a310400 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:36.588174 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-501b2b33-97d5-4ecb-9796-70ad0a310400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:36.596285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1135/2308] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:52:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:36.608842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:36.619749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91493202-0590-4157-95db-344f8c7b1c65 service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:36.629080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:36.629717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:36.630023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:36.630536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:36.639784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:36.641493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:36.652413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1174/2309] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:52:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:52:37.012831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:37.013153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:37.013424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:37.037810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:37.039034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:37.047155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1136/2310] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 19:52:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:37.060606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:37.074174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f93690-a872-448e-b306-9567a70204d9 service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:37.081696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:37.082578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:37.083001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:37.083322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:37.096913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:37.099501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:37.111118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1175/2311] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:52:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:52:42.669532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:42.686176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:42.688330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:42.704357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:42.783343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9dfd8-957c-4223-b029-9bff7da21998 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:42.791764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:42.792552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:42.793086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:42.793555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] PATH_INFO: `/v3/users/7bde6bd467c34da9a8f27f528735d85f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:42.795249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bde6bd467c34da9a8f27f528735d85f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:42.798935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:42.830541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4e025-6eb0-4475-a288-2e57eff66f82 service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:42.836053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:42.836696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:42.837020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:42.837356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] PATH_INFO: `/v3/users/299a8c2de0cf4f0aa1570c718fe04e52` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:42.838756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] RBAC: Authorizing `identity:delete_user(user_id=299a8c2de0cf4f0aa1570c718fe04e52)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:42.841535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:42.879513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bde6bd467c34da9a8f27f528735d85f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.889402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:42.890727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bde6bd467c34da9a8f27f528735d85f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.891391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bde6bd467c34da9a8f27f528735d85f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.892194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bde6bd467c34da9a8f27f528735d85f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.908870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-38da6a9c-bebd-4d27-b7de-75f9fa233b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bde6bd467c34da9a8f27f528735d85f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.914041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299a8c2de0cf4f0aa1570c718fe04e52'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.924688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299a8c2de0cf4f0aa1570c718fe04e52'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.925492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1137/2312] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:42 2022] DELETE /identity/v3/users/7bde6bd467c34da9a8f27f528735d85f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:42.929494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299a8c2de0cf4f0aa1570c718fe04e52'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.930243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299a8c2de0cf4f0aa1570c718fe04e52'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.933762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:42.934047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:42.934308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:42.938985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-49e45a97-24ca-4b36-8f2e-1901562b783f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299a8c2de0cf4f0aa1570c718fe04e52'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:42.955691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:42.957240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:42.959144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1176/2313] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:42 2022] DELETE /identity/v3/users/299a8c2de0cf4f0aa1570c718fe04e52 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:42.965446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:42.965764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:42.966064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:42.984903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:42.986141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:43.022505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1138/2314] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:43.032435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:43.034003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:43.049431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1177/2315] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:43.059305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:43.060753 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:43.081037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:43.099593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a237663-0452-4379-bf38-78f29f7661d6 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:43.103384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:43.103993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:43.104267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:43.104534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:43.113930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:43.115378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:43.121454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58faa59-bd6d-496e-900c-3b47765d89d5 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:43.125119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:43.125617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1139/2316] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:43.125758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:43.126006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:43.126271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:43.136260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:43.137747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:43.148390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1178/2317] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:52:43.590986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:43.593432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:43.609326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212495c7-b48e-41eb-b592-e436789bee2a service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:43.616225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:43.616771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:43.616970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:43.617208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] PATH_INFO: `/v3/projects/62ad20021ed0483186dad6591ba5ce67` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:43.618386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ad20021ed0483186dad6591ba5ce67)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:43.620325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:43.685235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ad20021ed0483186dad6591ba5ce67', 'request_id': 'req-c4268734-1e7f-4271-9e20-57f7b45bf806'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.693177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ad20021ed0483186dad6591ba5ce67', 'request_id': 'req-c4268734-1e7f-4271-9e20-57f7b45bf806'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.703032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ad20021ed0483186dad6591ba5ce67', 'request_id': 'req-c4268734-1e7f-4271-9e20-57f7b45bf806'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.712473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:43.713930 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:43.716355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] The token cache is being invalidate because project 62ad20021ed0483186dad6591ba5ce67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:43.716724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.717479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.719320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1140/2318] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:43 2022] DELETE /identity/v3/projects/62ad20021ed0483186dad6591ba5ce67 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:43.727492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e4df3-35b9-414b-beb1-bed610ce810a service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:43.831381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:43.859400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:43.860071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:43.860567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:43.860850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] PATH_INFO: `/v3/projects/1a34d8d2a04046d1b16c778106a52f5e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:43.863158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a34d8d2a04046d1b16c778106a52f5e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:43.866826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:43.957496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a34d8d2a04046d1b16c778106a52f5e', 'request_id': 'req-3886378b-ea08-4a85-a9c7-96040236586b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.957847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a34d8d2a04046d1b16c778106a52f5e', 'request_id': 'req-3886378b-ea08-4a85-a9c7-96040236586b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.966317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a34d8d2a04046d1b16c778106a52f5e', 'request_id': 'req-3886378b-ea08-4a85-a9c7-96040236586b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.977920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] The token cache is being invalidate because project 1a34d8d2a04046d1b16c778106a52f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:43.978268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.978931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:43.980460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1179/2319] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:43 2022] DELETE /identity/v3/projects/1a34d8d2a04046d1b16c778106a52f5e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:43.984493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:43.986453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:44.001373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4268734-1e7f-4271-9e20-57f7b45bf806 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:44.079303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:44.080061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:44.080332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:44.080669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] PATH_INFO: `/v3/users/c5183fbd21824f2995253f615369486f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:44.081967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5183fbd21824f2995253f615369486f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:44.084725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:44.136806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5183fbd21824f2995253f615369486f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.145298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5183fbd21824f2995253f615369486f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.145923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5183fbd21824f2995253f615369486f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.146531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5183fbd21824f2995253f615369486f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.156655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:44.159092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-fa169c72-f223-40d0-b58f-a4c1f2d7cdb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5183fbd21824f2995253f615369486f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.173429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1141/2320] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:43 2022] DELETE /identity/v3/users/c5183fbd21824f2995253f615369486f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:44.546611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:44.549454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:44.565907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886378b-ea08-4a85-a9c7-96040236586b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:44.673284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:44.673284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:44.673284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:44.673284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] PATH_INFO: `/v3/projects/a2f48805059949248392aa1aa929fb43` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:44.673663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f48805059949248392aa1aa929fb43)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:44.674163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:44.734609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f48805059949248392aa1aa929fb43', 'request_id': 'req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.742240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f48805059949248392aa1aa929fb43', 'request_id': 'req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.751953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f48805059949248392aa1aa929fb43', 'request_id': 'req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.764776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] The token cache is being invalidate because project a2f48805059949248392aa1aa929fb43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:44.765355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.766091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29364ed6-4e12-42a2-aa13-87ba8a7b4e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:44.767674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1180/2321] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:44 2022] DELETE /identity/v3/projects/a2f48805059949248392aa1aa929fb43 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:44.772962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:44.773483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:44.773574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:44.800102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:44.802135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:44.905040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1142/2322] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:44.912761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:44.913051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:44.913250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:44.928129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:44.942834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:44.945020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:44.971015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1181/2323] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:44.975527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:44.987880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a2329-9676-4908-86e7-4a85366c61bb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:44.992906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:44.993639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:44.993981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:44.994320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:44.995160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:44.995535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:44.997641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.018515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1143/2324] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:45.028583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.034825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbc4f9-0a0a-4f65-bfaa-2f82cec97a2e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.044225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.045063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.045345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.045616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.046271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.049033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.100633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1182/2325] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:45.105065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.117420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d907d-dcc6-4103-ba46-ebade3700f8f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.196252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.196936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.197268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.197486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.198126 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.200073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.230925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1144/2326] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:45.234927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.245584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f554a6d-9365-447d-a72a-5669273d5e96 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.250477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.251109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.251364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.251627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.252266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.253945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.264676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1183/2327] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:45.269754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.283744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b8ed6-0a53-4c7e-8a5a-a0ca65dff825 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.289099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.289700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.289971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.290260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.290887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.292506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a7995-cb34-44d3-b5d0-4df3c48843f9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.303177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1145/2328] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:45.307932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.331956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ea68c-4fc2-44fc-9f10-f91fa0984eb6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.339910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.341396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.342048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.342645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] PATH_INFO: `/v3/projects/43bb62ad51834375abaa2290b1977a10/users/f26e852296664ba2b52fa5702ca70326/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.370235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] RBAC: Authorizing `identity:create_grant(project_id=43bb62ad51834375abaa2290b1977a10, user_id=f26e852296664ba2b52fa5702ca70326, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.375988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.397114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1184/2329] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:45 2022] PUT /identity/v3/projects/43bb62ad51834375abaa2290b1977a10/users/f26e852296664ba2b52fa5702ca70326/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:45.416783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.417094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.417364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.447103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] MFA Rules not processed for user `f26e852296664ba2b52fa5702ca70326`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:45.448328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:45.493241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:45.520977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1146/2330] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:45.525847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.590687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b9771-8b4d-4d3f-8767-a38c2392805f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.595997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.596675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.597008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.597222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.597954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.599999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.638460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1185/2331] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:45.640218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.651873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe6227-4eba-4f47-9d15-48a346af06c7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.713782 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.714419 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.714788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.714954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.715592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.717591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.742156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1147/2332] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:45.746600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.757642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7786e08-0509-48da-bc34-e6933dde836d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.762774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.763379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.763654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.763920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.764558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.766215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.779034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1186/2333] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:45.783159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.794368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7afd-bdf0-452f-95be-45beaef2e848 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.799353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.800080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.800363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.800631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.801323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.803001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cb106f-268e-47e8-bdd7-e4c50cba35c5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.813251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1148/2334] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:45.817671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.828250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4792fb9b-7781-46b7-85a4-1cbd7348d77a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.833320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.833972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.834332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.834613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] PATH_INFO: `/v3/projects/3c85e77b72074db98b6ec175a1f37ad1/users/054ca4040a86405e8cf9e9f58235434f/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.849626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c85e77b72074db98b6ec175a1f37ad1, user_id=054ca4040a86405e8cf9e9f58235434f, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.852311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.865724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1187/2335] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:45 2022] PUT /identity/v3/projects/3c85e77b72074db98b6ec175a1f37ad1/users/054ca4040a86405e8cf9e9f58235434f/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:45.876257 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.876538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.876809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.905358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] MFA Rules not processed for user `054ca4040a86405e8cf9e9f58235434f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:45.906496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:45.975614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1149/2336] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:45.980472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.041625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:46.051972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a24c-9bc8-488d-a62e-25fa6aed2ce7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.057469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.058101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.058380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.058649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.059282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.061362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.097403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1188/2337] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:46.099730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.111606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129068dd-e97d-42ab-8df8-c9c8da605bdd None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.168297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:46.184394 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.185144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.185485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.185823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.186995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.189014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.213619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1150/2338] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:46 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:46.222372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.234575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efad9aa-b544-472c-bbd1-ae57166f51eb admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.241585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.242217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.242496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.242816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.243549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.245430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.258114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1189/2339] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:46.262315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.279186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9a208-f8ed-4f71-8954-3fe0f6ca9b29 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.289568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.289568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.289568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.289568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] PATH_INFO: `/v3/projects/9ce69bc587df47338ee1e00a1933a1aa/users/8f32508c847b440ab5f541cb34ba2fe6/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.314005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce69bc587df47338ee1e00a1933a1aa, user_id=8f32508c847b440ab5f541cb34ba2fe6, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.317105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.344275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1151/2340] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:46 2022] PUT /identity/v3/projects/9ce69bc587df47338ee1e00a1933a1aa/users/8f32508c847b440ab5f541cb34ba2fe6/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:46.348658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.367093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecbd3a-3eb1-4046-a66e-22f923c66d57 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.445568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.446558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.447049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.447216 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.447981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.449713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1578c-1957-40b2-ba63-84b55152b243 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.475607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1190/2341] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:46.479923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.490713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa691f39-46e1-4412-8faa-170c1134cc73 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.500493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.501289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.501659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.502067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] PATH_INFO: `/v3/projects/9ce69bc587df47338ee1e00a1933a1aa/users/8f32508c847b440ab5f541cb34ba2fe6/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.504843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce69bc587df47338ee1e00a1933a1aa, user_id=8f32508c847b440ab5f541cb34ba2fe6, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.507522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.520960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1152/2342] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:46 2022] PUT /identity/v3/projects/9ce69bc587df47338ee1e00a1933a1aa/users/8f32508c847b440ab5f541cb34ba2fe6/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:46.531200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaeec2-fab2-4a36-abee-111ca593ebad None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.531539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaeec2-fab2-4a36-abee-111ca593ebad None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.531748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeaeec2-fab2-4a36-abee-111ca593ebad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.566647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-dfeaeec2-fab2-4a36-abee-111ca593ebad None None] MFA Rules not processed for user `8f32508c847b440ab5f541cb34ba2fe6`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:46.566647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dfeaeec2-fab2-4a36-abee-111ca593ebad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.649675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1191/2343] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:46.669046 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.672473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.754581 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ca0a1-bf42-4596-b5a6-27afb27f1efe admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.758431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.759052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.759330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.759601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.832623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.834649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.846073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1153/2344] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:48.115840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.116345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.116439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.135996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] MFA Rules not processed for user `a1c9d4af0a98406692e1d79293844c68`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:48.137268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:48.187894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:48.204014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1192/2345] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:48.504524 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.506138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:48.518162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a8f77d-6b0e-4e42-a388-4d8b1faecac6 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.544125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.545107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.545699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.546150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.610401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:48.614567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.616696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.627881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1154/2346] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:52:49.291448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:49.302518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c6b-a7ce-4cf2-b94d-7c3e1a89678e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:49.306995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:49.307995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:49.308334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:49.308666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:49.319246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:49.320731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3df5e5-58af-4484-acb1-602ac8618309 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:49.331521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1193/2347] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:53.386967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:53.496165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df23cf-d1c4-421e-adfc-33f9baf553d4 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:53.512147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:53.514207 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:53.514712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:53.515249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] PATH_INFO: `/v3/users/f26e852296664ba2b52fa5702ca70326` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:53.516913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f26e852296664ba2b52fa5702ca70326)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:53.519850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:53.608414 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f26e852296664ba2b52fa5702ca70326'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:53.623880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f26e852296664ba2b52fa5702ca70326'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:53.624550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f26e852296664ba2b52fa5702ca70326'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:53.625431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f26e852296664ba2b52fa5702ca70326'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:53.659112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f26e852296664ba2b52fa5702ca70326'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:53.686109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1155/2348] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:53 2022] DELETE /identity/v3/users/f26e852296664ba2b52fa5702ca70326 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:53.691534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:53.692060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:53.692060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:53.721346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:53.724040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:53.833770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1194/2349] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:53.852349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:53.861583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:53.945793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d746a-4115-4f39-8fb9-85a8478e6720 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:53.949635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:53.950544 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:53.950831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:53.951107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:53.962664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:53.964786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:53.983551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1156/2350] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:54.335765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:54.337966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:54.369552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2aeb4-c09f-4779-b9d5-0df004d429a9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:54.392745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:54.394970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:54.395410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:54.395904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] PATH_INFO: `/v3/projects/43bb62ad51834375abaa2290b1977a10` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:54.398981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43bb62ad51834375abaa2290b1977a10)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:54.405843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:54.473610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43bb62ad51834375abaa2290b1977a10', 'request_id': 'req-0efa480c-eb1b-4432-bf69-f8ece3b28224'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.490767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43bb62ad51834375abaa2290b1977a10', 'request_id': 'req-0efa480c-eb1b-4432-bf69-f8ece3b28224'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.490767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43bb62ad51834375abaa2290b1977a10', 'request_id': 'req-0efa480c-eb1b-4432-bf69-f8ece3b28224'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.502853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] The token cache is being invalidate because project 43bb62ad51834375abaa2290b1977a10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:54.503496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.504361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.506143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1195/2351] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:54 2022] DELETE /identity/v3/projects/43bb62ad51834375abaa2290b1977a10 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:54.510622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:54.521359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f6c02-24f8-4ff8-9e3b-05c312851a61 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:54.537579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:54.598367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:54.598979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:54.599289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:54.599514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] PATH_INFO: `/v3/users/054ca4040a86405e8cf9e9f58235434f` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:54.600689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=054ca4040a86405e8cf9e9f58235434f)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:54.602772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:54.670135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '054ca4040a86405e8cf9e9f58235434f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.679654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '054ca4040a86405e8cf9e9f58235434f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.680349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '054ca4040a86405e8cf9e9f58235434f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.681677 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '054ca4040a86405e8cf9e9f58235434f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.699413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '054ca4040a86405e8cf9e9f58235434f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:54.712715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1157/2352] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:54 2022] DELETE /identity/v3/users/054ca4040a86405e8cf9e9f58235434f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:55.232278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:55.234825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:55.266741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa480c-eb1b-4432-bf69-f8ece3b28224 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:55.384870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:55.385825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:55.386109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:55.386411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] PATH_INFO: `/v3/projects/3c85e77b72074db98b6ec175a1f37ad1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:55.387646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c85e77b72074db98b6ec175a1f37ad1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:55.392599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:55.405014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:55.469468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c85e77b72074db98b6ec175a1f37ad1', 'request_id': 'req-ec04c704-52bf-4f90-a389-442a188ad0c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.477183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c85e77b72074db98b6ec175a1f37ad1', 'request_id': 'req-ec04c704-52bf-4f90-a389-442a188ad0c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.487792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c85e77b72074db98b6ec175a1f37ad1', 'request_id': 'req-ec04c704-52bf-4f90-a389-442a188ad0c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.505374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] The token cache is being invalidate because project 3c85e77b72074db98b6ec175a1f37ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:55.505705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.506401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.508309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1196/2353] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:55 2022] DELETE /identity/v3/projects/3c85e77b72074db98b6ec175a1f37ad1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:55.512308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:55.517478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:55.531225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efde2f-22c6-48aa-ae40-337a81680131 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:55.600834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:55.601548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:55.601822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:55.602101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] PATH_INFO: `/v3/users/8f32508c847b440ab5f541cb34ba2fe6` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:55.603339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f32508c847b440ab5f541cb34ba2fe6)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:55.605367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:55.658946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:55.671063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f32508c847b440ab5f541cb34ba2fe6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.679161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f32508c847b440ab5f541cb34ba2fe6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.679910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f32508c847b440ab5f541cb34ba2fe6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.680720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f32508c847b440ab5f541cb34ba2fe6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.695390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-f61479e3-faca-4482-9a05-17b8af3ecff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f32508c847b440ab5f541cb34ba2fe6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:55.710156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1158/2354] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:55 2022] DELETE /identity/v3/users/8f32508c847b440ab5f541cb34ba2fe6 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:56.013761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.015325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:56.039191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04c704-52bf-4f90-a389-442a188ad0c9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.105373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.106021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.106301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.106575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] PATH_INFO: `/v3/projects/9ce69bc587df47338ee1e00a1933a1aa` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.107722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce69bc587df47338ee1e00a1933a1aa)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.109832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.163124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce69bc587df47338ee1e00a1933a1aa', 'request_id': 'req-62817315-8fef-4628-9d31-2ca54849e518'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:56.170293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce69bc587df47338ee1e00a1933a1aa', 'request_id': 'req-62817315-8fef-4628-9d31-2ca54849e518'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:56.181978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce69bc587df47338ee1e00a1933a1aa', 'request_id': 'req-62817315-8fef-4628-9d31-2ca54849e518'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:56.197607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] The token cache is being invalidate because project 9ce69bc587df47338ee1e00a1933a1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:56.197976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:56.198789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62817315-8fef-4628-9d31-2ca54849e518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:56.201056 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1197/2355] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:56 2022] DELETE /identity/v3/projects/9ce69bc587df47338ee1e00a1933a1aa => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:56.209307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.209307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.209307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.230308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:56.231657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:56.322824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1159/2356] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:56.324448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.324966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.325098 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.348298 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:56.349623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:56.359152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1198/2357] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:56.363564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.373841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01bed70-befb-4f20-8840-9f9b890ff5da None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.379660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.380395 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.380734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.381249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.382010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:56.382310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.383963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.400290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1160/2358] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:56.405939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.422834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb18ec7-cd05-472e-a00a-63b502446143 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.429831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.430409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.430688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.430964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.431603 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.433566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.457024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:56.467093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1199/2359] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:56 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:56.468566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.480259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4806b-ed26-4e4f-972d-3e681638a5fe admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.544665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.545495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.545611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.545838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.546506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.548453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.574393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1161/2360] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:56 2022] POST /identity/v3/users => generated 295 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:56.578454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.591195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58334c3f-cf25-471c-ac0f-e20e1717852e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.596469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.597426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.598049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.598329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.599011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.600620 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.612932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1200/2361] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:56.615822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.633603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aabe6b7-b3e5-4980-89e5-18e03c8f8b37 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.639355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.639990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.640280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.640536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.641220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.642836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8904bb-d1c5-4c20-9b5f-2f733496510b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.655314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1162/2362] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:56.661019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.672967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2431-c9f1-40c2-aaa0-ffcb4037fa72 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.678311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.679102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.679274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.679552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] PATH_INFO: `/v3/projects/a11ae3adb65b4213a3fa613e3aa810f1/users/148af185474f4a7bb823088e5e75b1c9/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.698402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11ae3adb65b4213a3fa613e3aa810f1, user_id=148af185474f4a7bb823088e5e75b1c9, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.702245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.716183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1201/2363] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:56 2022] PUT /identity/v3/projects/a11ae3adb65b4213a3fa613e3aa810f1/users/148af185474f4a7bb823088e5e75b1c9/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:56.727016 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0131-bc3c-48f2-b397-dba470f835c3 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.727381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0131-bc3c-48f2-b397-dba470f835c3 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.727845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ef0131-bc3c-48f2-b397-dba470f835c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.766066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c5ef0131-bc3c-48f2-b397-dba470f835c3 None None] MFA Rules not processed for user `148af185474f4a7bb823088e5e75b1c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:56.767306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c5ef0131-bc3c-48f2-b397-dba470f835c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:56.776731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:56.838629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1163/2364] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:52:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:56.849819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:56.851711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:56.915361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a3026-11a2-4b75-8872-fe5b0068248d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:56.919666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:56.920282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:56.920557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:56.920823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:56.931022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:56.932506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:56.942460 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1202/2365] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:03.539233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.539712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.539801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.560636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:03.561880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.622215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1164/2366] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:03.635609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:03.637248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.735042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3195e60a-0571-4575-a107-b5e78b8268aa service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:03.740923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:03.741508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.741781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.743140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.756974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:03.758713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04514c8-945b-43d9-9b9a-fd1b9d0f9b9d service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:03.774946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1203/2367] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:53:13.615561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=48 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:13.768389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=75,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:17.899146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:17.966832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886da975-e074-43d4-b87d-b67736d46ed1 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:17.971630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:17.972273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:17.972551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:17.972800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] PATH_INFO: `/v3/users/148af185474f4a7bb823088e5e75b1c9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:17.974029 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=148af185474f4a7bb823088e5e75b1c9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:17.975948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:18.043238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '148af185474f4a7bb823088e5e75b1c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:18.053691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '148af185474f4a7bb823088e5e75b1c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:18.054409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '148af185474f4a7bb823088e5e75b1c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:18.055127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '148af185474f4a7bb823088e5e75b1c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:18.076953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '148af185474f4a7bb823088e5e75b1c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:18.107750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1165/2368] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:17 2022] DELETE /identity/v3/users/148af185474f4a7bb823088e5e75b1c9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:18.113911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:18.114581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:18.115049 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:18.141213 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:18.142449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:18.230054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1204/2369] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:18.241086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:18.243357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:18.327848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53765022-230f-415b-b583-5a7d5a36aaf0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:18.332526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:18.333188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:18.333632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:18.333905 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:18.361425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:18.363779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6549c1ee-d3c8-42f5-a74b-4459671c4944 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:18.378013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1166/2370] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:18.939383 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:18.941251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:18.962508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40688c5d-712f-4a3f-836b-d1daa532f141 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:18.968665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:18.969772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:18.970417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:18.971042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] PATH_INFO: `/v3/projects/a11ae3adb65b4213a3fa613e3aa810f1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:18.973075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11ae3adb65b4213a3fa613e3aa810f1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.012377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.138825 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11ae3adb65b4213a3fa613e3aa810f1', 'request_id': 'req-10d48073-bd09-49a4-bd3c-9dd4c65a61da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:19.160035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11ae3adb65b4213a3fa613e3aa810f1', 'request_id': 'req-10d48073-bd09-49a4-bd3c-9dd4c65a61da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:19.179230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11ae3adb65b4213a3fa613e3aa810f1', 'request_id': 'req-10d48073-bd09-49a4-bd3c-9dd4c65a61da'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:19.206737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] The token cache is being invalidate because project a11ae3adb65b4213a3fa613e3aa810f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:19.207345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:19.208362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-10d48073-bd09-49a4-bd3c-9dd4c65a61da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:19.210806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1205/2371] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:18 2022] DELETE /identity/v3/projects/a11ae3adb65b4213a3fa613e3aa810f1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:19.216141 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.217248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.217248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.241404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:19.242672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:19.262210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:19.319480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1167/2372] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:19.328250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.328532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.328802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.355811 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:19.356992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:19.365577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1206/2373] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:19.370064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.380826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a064-1395-4e6a-ba3e-45a2290be720 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.387882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.388863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.389365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.389639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.390300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:19.390600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.392266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.405251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1168/2374] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:53:19.411927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.422767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb62b2a-5999-4a3f-a6b1-94517a8d31a9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.427946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.428653 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.429122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.429428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.430075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.432652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.468692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1207/2375] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:19 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:19.473047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.483730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9c750-ad98-4bd1-8187-e1e6d697e96f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.563050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.565238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.565238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.565715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.566642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.568835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.613436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1169/2376] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:19 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:19.618386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.635987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b94a3b7-f64c-4797-91da-8742ee889c06 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.647286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.648408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.648716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.649074 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.650043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.652339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.668477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1208/2377] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:53:19.673380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.688945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a045e59-0f05-4c09-9498-5834ef09e78c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.702705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.703741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.704264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.704733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.705963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.708450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c254fb3c-8682-4451-b6cd-87c2981d2c95 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.728942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1170/2378] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:53:19.735163 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.751564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a3ddc-c58e-4db9-8fbf-fd758260eb31 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:19.761322 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:19.762062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.762351 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.762626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] PATH_INFO: `/v3/projects/845d6ae8d6434c4c96e24cf39f309d23/users/771ba65bd2974845ac79d04ad3634c96/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.802910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] RBAC: Authorizing `identity:create_grant(project_id=845d6ae8d6434c4c96e24cf39f309d23, user_id=771ba65bd2974845ac79d04ad3634c96, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:19.805727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:19.827539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1209/2379] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:19 2022] PUT /identity/v3/projects/845d6ae8d6434c4c96e24cf39f309d23/users/771ba65bd2974845ac79d04ad3634c96/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:19.832805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b428a7-cb43-4c12-af35-3c758da4c90a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:19.833359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b428a7-cb43-4c12-af35-3c758da4c90a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:19.833778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b428a7-cb43-4c12-af35-3c758da4c90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:19.862523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-88b428a7-cb43-4c12-af35-3c758da4c90a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:19.863745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-88b428a7-cb43-4c12-af35-3c758da4c90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:19.966648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1171/2380] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:19.978561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:19.980806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:20.009289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:20.076488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a0e3c-eccd-4a90-ae36-3365ecaa882b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:20.080559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:20.081250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:20.081551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:20.081795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:20.094504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:20.096880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1addf7a-8dd9-4229-807c-dd9a347937b5 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:20.111234 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1210/2381] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:21.090813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.092274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.093138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.096526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1172/2382] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:53:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:21.103867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2bf1e-7d70-4c1c-8c94-ef46e22311a2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.104685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2bf1e-7d70-4c1c-8c94-ef46e22311a2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.104685 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed2bf1e-7d70-4c1c-8c94-ef46e22311a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.148644 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bed2bf1e-7d70-4c1c-8c94-ef46e22311a2 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:21.150060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bed2bf1e-7d70-4c1c-8c94-ef46e22311a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:21.317658 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1211/2383] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:53:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:21.328879 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:21.332209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:21.461392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:21.533146 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d2d86-2f01-48fc-b6a2-39d7b15a47fb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:21.543602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b6415-7e34-4762-8759-0215be91097c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:21.545631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6415-7e34-4762-8759-0215be91097c service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.546530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6415-7e34-4762-8759-0215be91097c service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.548314 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b6415-7e34-4762-8759-0215be91097c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.573181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b6415-7e34-4762-8759-0215be91097c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:21.576624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b6415-7e34-4762-8759-0215be91097c service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:21.603885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1173/2384] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:21.701672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facf2d-fa37-42b1-b713-85215137a818 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.702648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facf2d-fa37-42b1-b713-85215137a818 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.703414 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94facf2d-fa37-42b1-b713-85215137a818 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.706342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1212/2385] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:53:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:21.712839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab1c8e-7bb5-4e1e-8f89-b140f1f07a27 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.713345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab1c8e-7bb5-4e1e-8f89-b140f1f07a27 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.713799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab1c8e-7bb5-4e1e-8f89-b140f1f07a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.782832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ffab1c8e-7bb5-4e1e-8f89-b140f1f07a27 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:21.784078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ffab1c8e-7bb5-4e1e-8f89-b140f1f07a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:21.812741 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1174/2386] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:53:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:21.824577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facf2d-fa37-42b1-b713-85215137a818 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:21.853741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94facf2d-fa37-42b1-b713-85215137a818 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:21.859564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:21.860211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:21.860489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:21.860758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:21.871227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:21.872709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee405c69-f851-49df-8dec-baf5dac2e3c9 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:21.885319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1213/2387] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:22.103465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229084-4f56-4312-b2b1-55c4a8eadbd5 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:22.104367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229084-4f56-4312-b2b1-55c4a8eadbd5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:22.104865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229084-4f56-4312-b2b1-55c4a8eadbd5 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:22.107053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1175/2388] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:53:22.117078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e84b-ca9d-4da2-bd9e-7ced5bc9bed0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:22.117566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e84b-ca9d-4da2-bd9e-7ced5bc9bed0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:22.117683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8e84b-ca9d-4da2-bd9e-7ced5bc9bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:22.170546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-64d8e84b-ca9d-4da2-bd9e-7ced5bc9bed0 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:22.171815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-64d8e84b-ca9d-4da2-bd9e-7ced5bc9bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:22.186334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1214/2389] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:53:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:22.209937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38229084-4f56-4312-b2b1-55c4a8eadbd5 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:22.230867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38229084-4f56-4312-b2b1-55c4a8eadbd5 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:22.237493 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:22.238915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:22.239369 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:22.239857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:22.261926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:22.263657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:22.278609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1176/2390] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:25.994577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:25.995048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:25.995143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.034271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] MFA Rules not processed for user `771ba65bd2974845ac79d04ad3634c96`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:26.035854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.113053 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1215/2391] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:26.114411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.115845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.117384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.129307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30fd9f-dae3-4b40-8b37-2fdf810aed3b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.130378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d536e08f-4dff-464b-bf66-cce7ee580729 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.133842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.134489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.134789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.134888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.135055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.135512 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.135680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.135776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.136051 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] PATH_INFO: `/v3/users/54485e238883496a9b27fe380d757b7a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.137223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54485e238883496a9b27fe380d757b7a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.137609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.139181 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.177951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1216/2392] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:26 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:26.183243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.192468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:26.223964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54485e238883496a9b27fe380d757b7a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:26.239424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54485e238883496a9b27fe380d757b7a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:26.240090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54485e238883496a9b27fe380d757b7a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:26.240707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54485e238883496a9b27fe380d757b7a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:26.260593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9613aa0a-2f9d-45ac-b6fd-10d6ec5c9515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54485e238883496a9b27fe380d757b7a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:26.275179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1177/2393] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:26 2022] DELETE /identity/v3/users/54485e238883496a9b27fe380d757b7a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:53:26.280385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.280851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.281261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.287538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6a7f9-7728-4433-90c0-1c3dd2aff548 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.303883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:26.305312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.356428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.357210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.357440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.357714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.358357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.360359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.382733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1178/2394] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:26.386803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1217/2395] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:26 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:26.389909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.391675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.391947 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.394064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.426073 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84416d4a-f67e-436b-b3b0-1a1c9ad79178 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.428420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:26.433434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.434821 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.435107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.435380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.436085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.438640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.457941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1218/2396] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:26.459586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.470687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d723-a806-4a1a-b4e7-ec99d169370b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.476252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad29877-2d24-4d26-baef-878c8e026c92 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.480947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.481214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.481654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.481832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.482024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.482115 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.482309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] PATH_INFO: `/v3/projects/042e05994c7a4667a52c35869ddaf68a/users/9b52efa07ab34e9aae801b2fb2f14ad0/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.482416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.495763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.497498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.501329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=042e05994c7a4667a52c35869ddaf68a, user_id=9b52efa07ab34e9aae801b2fb2f14ad0, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.505179 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.511674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1179/2397] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:53:26.526367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1219/2398] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:26 2022] PUT /identity/v3/projects/042e05994c7a4667a52c35869ddaf68a/users/9b52efa07ab34e9aae801b2fb2f14ad0/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:26.530776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.541509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593aee41-18ee-4046-938a-01ae59ccba89 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.615127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.615928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.616244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.616543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.617278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.619183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.635548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1180/2399] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:26.639521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.651835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435f71e-4e64-4aa6-b6d3-dd672aa358ff admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:26.661664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:26.662400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:26.662687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:26.662964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] PATH_INFO: `/v3/projects/042e05994c7a4667a52c35869ddaf68a/users/9b52efa07ab34e9aae801b2fb2f14ad0/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:26.665548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=042e05994c7a4667a52c35869ddaf68a, user_id=9b52efa07ab34e9aae801b2fb2f14ad0, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:26.668404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ddee0-9222-4701-97c7-717d4c2980d9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:26.681965 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1220/2400] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:26 2022] PUT /identity/v3/projects/042e05994c7a4667a52c35869ddaf68a/users/9b52efa07ab34e9aae801b2fb2f14ad0/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:26.964379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:26.967817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:26.987924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e813721-e0b4-48a4-95c9-cdaf2c2f0a99 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:27.072521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:27.073374 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:27.073883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:27.074280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] PATH_INFO: `/v3/projects/dbb030840abe441a964c521dc69f230e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:27.075635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb030840abe441a964c521dc69f230e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:27.077724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:27.145960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb030840abe441a964c521dc69f230e', 'request_id': 'req-6c385c15-5dcf-4d74-b46f-26f46398b04a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:27.154840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb030840abe441a964c521dc69f230e', 'request_id': 'req-6c385c15-5dcf-4d74-b46f-26f46398b04a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:27.170949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb030840abe441a964c521dc69f230e', 'request_id': 'req-6c385c15-5dcf-4d74-b46f-26f46398b04a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:27.177965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:27.202409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] The token cache is being invalidate because project dbb030840abe441a964c521dc69f230e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:27.202708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:27.203537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:27.205307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1181/2401] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:26 2022] DELETE /identity/v3/projects/dbb030840abe441a964c521dc69f230e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:53:31.561071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efe68-de2e-472d-914d-a741abfddaf2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:31.561652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efe68-de2e-472d-914d-a741abfddaf2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:31.561652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6efe68-de2e-472d-914d-a741abfddaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:31.597128 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-2b6efe68-de2e-472d-914d-a741abfddaf2 None None] MFA Rules not processed for user `9b52efa07ab34e9aae801b2fb2f14ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:31.598365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2b6efe68-de2e-472d-914d-a741abfddaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:31.659316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:31.701107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1221/2402] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:31.720653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:31.720653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:31.788290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c385c15-5dcf-4d74-b46f-26f46398b04a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:31.792333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:31.793191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:31.794068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:31.794387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:31.796222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:31.865588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:31.867499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:31.879740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1182/2403] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:32.041367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:32.042375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:32.042680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:32.063684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:32.065159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:32.105196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:32.106902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:32.175253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1222/2404] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:53:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:32.176512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:32.178839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:32.245908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b0736-625e-4b75-b239-c04476ba3d93 service glance] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:32.254347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:32.255792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:32.256026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:32.256299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:32.258431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:32.273945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:32.274153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f36ce-caf5-465f-b0d7-70b54e7788f6 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:32.275958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51b239-9d23-4701-9477-dae9f1c956b7 service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:32.281113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:32.282336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:32.282731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:32.283008 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:32.301897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1183/2405] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:53:32.302678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:32.305557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:32.315269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1223/2406] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:53:37.186830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083310-41f2-4a6f-9906-a0174e46aeb5 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:37.187943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083310-41f2-4a6f-9906-a0174e46aeb5 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:37.188469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4083310-41f2-4a6f-9906-a0174e46aeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:37.222328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b4083310-41f2-4a6f-9906-a0174e46aeb5 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:37.223764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b4083310-41f2-4a6f-9906-a0174e46aeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:37.232247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1184/2407] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:53:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:37.260245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:37.272354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba65d00-df02-4931-b3e5-058891c8b192 service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:37.278883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:37.279765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:37.280042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:37.280315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:37.291297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:37.292824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa9d628-3422-416f-93b5-341e18cad9ce service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:37.301947 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1224/2408] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:53:42.294622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.295880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.296347 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.331886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] MFA Rules not processed for user `771ba65bd2974845ac79d04ad3634c96`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:42.333218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:42.341280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1185/2409] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:53:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:47.298129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=74 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:52.254931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34d02-81ff-423e-9bde-078e7cac278b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.255592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34d02-81ff-423e-9bde-078e7cac278b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.255592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc34d02-81ff-423e-9bde-078e7cac278b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.279513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6dc34d02-81ff-423e-9bde-078e7cac278b None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:52.281100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6dc34d02-81ff-423e-9bde-078e7cac278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:52.290186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1225/2410] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:52.303462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:52.319344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62668afe-62f1-4b40-bf49-ead637941a79 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:52.331274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:52.331906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.332178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.332604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.342183 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:52.344049 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e867ff-ceaf-46bb-9d80-030fd4146720 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:52.353778 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1186/2411] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:54:02.276124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:02.350014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=1,keystone:SELECT=94 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:12.757197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:12.757744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:12.757744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:12.777986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] MFA Rules not processed for user `11c76c57183e406c87f434841fc13b43`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:12.779217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:12.840091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1226/2412] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:54:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:14.650631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:14.651196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:14.651196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:14.677299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:14.678654 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:14.693667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1187/2413] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 19:54:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:54:14.709241 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:14.723614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051bf25-9d21-4a26-9cd9-0433364fa307 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:14.730898 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:14.731630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:14.732105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:14.732313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:14.742240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:14.744111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a150fe-51a1-4db3-b692-cd5b9f5b6791 service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:14.754016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1227/2414] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:54:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:54:24.668099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:24.750403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=31 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:29.618258 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:29.619811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:29.681707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1065e1-7856-4394-8d1f-bf3e6bbeccfb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:29.685663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:29.686549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:29.686968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:29.687233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:29.700734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:29.702296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:29.715643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1188/2415] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:36.042468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccd3-ca4b-42a5-a838-9dd2a765c7e1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:36.043083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccd3-ca4b-42a5-a838-9dd2a765c7e1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:36.043083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfccd3-ca4b-42a5-a838-9dd2a765c7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:36.062476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e2dfccd3-ca4b-42a5-a838-9dd2a765c7e1 None None] MFA Rules not processed for user `a1c9d4af0a98406692e1d79293844c68`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:36.063676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e2dfccd3-ca4b-42a5-a838-9dd2a765c7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:36.127026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1228/2416] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:54:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:39.708624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=26 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:46.118907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=25 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:09.855002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:09.863971 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364eca77-8ec5-4968-aa16-425ba002ef63 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:09.868999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:09.869660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:09.869941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:09.870210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:09.872162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:09.947770 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:09.949323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:09.961078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1189/2417] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:16.822955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:16.823672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:16.823672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:16.845380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] MFA Rules not processed for user `dbea9a6a35c6445a9ab76ee7278abda4`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:16.845622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:16.847454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:16.865330 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d15f0e-8ec7-4af3-83a3-c3267fbbdcd1 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:16.869486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:16.870101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:16.870561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:16.870979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:16.873052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:16.887508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:16.889034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:16.900473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1190/2418] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:55:16.908211 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1229/2419] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:17.270583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:17.279530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf9a4-497b-463c-b9c1-81fa35fcaf12 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:17.283488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:17.284074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:17.284388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:17.284680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:17.286750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:17.296360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:17.297745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:17.306050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1191/2420] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:18.641036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:18.655884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a315e3e-a165-4fa8-bf86-633b91e11848 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:18.663420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:18.664080 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:18.664353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:18.664618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:18.666510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:18.684597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:18.686543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:18.699102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1230/2421] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:19.951787 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:19.953453 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:20.042573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6339dd-6358-4361-b99b-bdc668119a15 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:20.048080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:20.048824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:20.049062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:20.049280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] PATH_INFO: `/v3/users/11c76c57183e406c87f434841fc13b43` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:20.051068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c76c57183e406c87f434841fc13b43)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:20.053300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:20.105750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c76c57183e406c87f434841fc13b43'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.115402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c76c57183e406c87f434841fc13b43'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.116274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c76c57183e406c87f434841fc13b43'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.117081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c76c57183e406c87f434841fc13b43'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.123325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:20.132768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c76c57183e406c87f434841fc13b43'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.146161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1192/2422] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:55:19 2022] DELETE /identity/v3/users/11c76c57183e406c87f434841fc13b43 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:20.324916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:20.336038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a1e1a-f7f3-4c69-b4e1-7ebca29445c5 service placement] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:20.400055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:20.400606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:20.400915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:20.401175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] PATH_INFO: `/v3/projects/fa70232219304082b5ee608a7ea61c22` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:20.402348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa70232219304082b5ee608a7ea61c22)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:20.404617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:20.453783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa70232219304082b5ee608a7ea61c22', 'request_id': 'req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.461114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa70232219304082b5ee608a7ea61c22', 'request_id': 'req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.469435 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa70232219304082b5ee608a7ea61c22', 'request_id': 'req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.480570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] The token cache is being invalidate because project fa70232219304082b5ee608a7ea61c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:55:20.481196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.482108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:20.493757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1231/2423] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:55:20 2022] DELETE /identity/v3/projects/fa70232219304082b5ee608a7ea61c22 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:24.457156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:24.458818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:24.534975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba50d25-750f-4bdd-b325-911c7b3acad0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:24.604087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:24.604700 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:24.605089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:24.605362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:24.607384 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:24.674724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:24.676478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a44771-a5ad-44de-ae2f-e9f092ba94e1 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:24.685862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1193/2424] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:28.237919 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:28.239494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:28.275385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:28.334337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d039cc8-d307-4b97-8ab7-9c2b682e48ff admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:28.339357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:28.340153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:28.340627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:28.341238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:28.343747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:28.357584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:28.359397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:28.377449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1232/2425] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:30.428165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:30.428691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:30.428691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:30.449211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] MFA Rules not processed for user `e60df169b5dc4aa59b5b0246544ef2b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:30.450430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:30.459724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:30.510531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1194/2426] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:31.365870 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:31.367340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:31.398209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab35846-448e-4b9c-a867-dcc32637ffc2 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:31.404112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:31.405174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:31.405610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:31.406146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:31.408481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:31.422402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:31.424077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:31.437348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1233/2427] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:34.672530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:34.674221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:34.693249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57afd11-ed2a-4f96-8d7a-0c26d7b80b7b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:34.699832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:34.700440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:34.700718 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:34.701023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:34.702896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:34.763815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:34.765446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:34.775227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1195/2428] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:41.431776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=22 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:44.770288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=47 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:45.078323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:45.090795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c85668-eeaa-4086-a4c3-d0bed86e6c15 service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:45.096992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:45.097727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.098078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.098419 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.100318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.115612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:45.117261 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:45.128557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1234/2429] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:54.264539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:54.274485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7dae8-853b-42cc-9b93-a16eb58a9dcc service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:54.279205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:54.279811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:54.280065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:54.280335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:54.282388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:54.347337 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:54.348783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:54.358874 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1196/2430] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:55.123785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:57.674911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:57.677380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:57.692377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50cca-1ebf-4804-bfd8-ee1d7d67189d service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:57.697473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:57.698110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:57.698387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:57.698660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] PATH_INFO: `/v3/users/a1c9d4af0a98406692e1d79293844c68` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:57.699790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1c9d4af0a98406692e1d79293844c68)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:57.702372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:57.763832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1c9d4af0a98406692e1d79293844c68'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:57.774705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1c9d4af0a98406692e1d79293844c68'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:57.775859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1c9d4af0a98406692e1d79293844c68'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:57.776563 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1c9d4af0a98406692e1d79293844c68'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:57.783912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-130a51fd-58f8-48c1-a13a-a52fb3a75a63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1c9d4af0a98406692e1d79293844c68'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:57.802208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1235/2431] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:55:57 2022] DELETE /identity/v3/users/a1c9d4af0a98406692e1d79293844c68 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:57.983075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:57.993332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a966b50e-eccf-4d92-a058-d332d39a389f service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.066173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.066816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.068335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.068335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] PATH_INFO: `/v3/projects/592bbccef3ef42ccbc7e2c3dc5bb86de` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.068826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] RBAC: Authorizing `identity:delete_project(project_id=592bbccef3ef42ccbc7e2c3dc5bb86de)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.070814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.119177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '592bbccef3ef42ccbc7e2c3dc5bb86de', 'request_id': 'req-475362fc-22bc-4d1d-918a-f55811c27148'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:58.125627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '592bbccef3ef42ccbc7e2c3dc5bb86de', 'request_id': 'req-475362fc-22bc-4d1d-918a-f55811c27148'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:58.135205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '592bbccef3ef42ccbc7e2c3dc5bb86de', 'request_id': 'req-475362fc-22bc-4d1d-918a-f55811c27148'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:58.146169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] The token cache is being invalidate because project 592bbccef3ef42ccbc7e2c3dc5bb86de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:55:58.146450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:58.147063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-475362fc-22bc-4d1d-918a-f55811c27148 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:55:58.148593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1197/2432] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:55:57 2022] DELETE /identity/v3/projects/592bbccef3ef42ccbc7e2c3dc5bb86de => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:58.159134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.159446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.159720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.179877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:58.181112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:58.248678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1236/2433] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:58.256411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.256694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.257000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.277914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:58.279104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:58.287731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1198/2434] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:58.293981 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.304555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceef7c86-9a6d-402f-8d95-e53ca78fbee2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.309997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.310604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.310875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.311165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.311806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:55:58.312106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.313748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.325539 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1237/2435] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:55:58.330105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.339729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700a33e-dcef-4934-8f69-b71c7aac57c0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.345129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.345723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.345998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.346261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.346889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.348762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.368259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:58.395877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1199/2436] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:55:58.397984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.409260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efa9b1-e0ba-4c30-aee7-6c7a7bde4147 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.472482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.473130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.473404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.473668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.474294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.476202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.484204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:58.501546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1238/2437] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:55:58.505919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.521324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da4d94e-bbb2-4d3b-8e1f-56c819d0221d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.528175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.528729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.529048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.529320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.529960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.531589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.542208 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1200/2438] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:55:58.546175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.557212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ad98c7-bb85-4d9a-92c3-ab30cf7e06b0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.563193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.563794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.564064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.564332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.564999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.566591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de8961-eb63-4471-9de4-18e4ade9d0b1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.578290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1239/2439] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:55:58.582773 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.593250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd783a8-6d33-404c-97a8-8b0772263094 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.604153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.604825 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.605150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.605423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] PATH_INFO: `/v3/projects/d9249d632ba24268ab9c2f5140c6ce25/users/e09ca7517c844f2ea7b4541b6e902037/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.621513 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9249d632ba24268ab9c2f5140c6ce25, user_id=e09ca7517c844f2ea7b4541b6e902037, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.624178 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.639346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1201/2440] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:55:58 2022] PUT /identity/v3/projects/d9249d632ba24268ab9c2f5140c6ce25/users/e09ca7517c844f2ea7b4541b6e902037/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:55:58.650834 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.651121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.651393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.683196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] MFA Rules not processed for user `e09ca7517c844f2ea7b4541b6e902037`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:58.684585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:58.752280 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1240/2441] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:58.756983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.822496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0523a84-e688-44ba-913d-67e66378b25e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.827902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.828558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.828759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.829089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.829720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.831611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.867474 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1202/2442] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:55:58.871978 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.884319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de045665-b226-42bd-9df4-9a16f0281d72 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.960203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.960818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.961126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.961397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.962024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.963938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.987092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1241/2443] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:55:58 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:55:58.992830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.004224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167649be-c3dc-4591-890e-b9bb886dc2d9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.009234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.009831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.010096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.010355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.011213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.012828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.027739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1203/2444] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:55:59.029655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.040424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b77d3-2f7d-40ed-87fc-69f6f4ef9424 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.045612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.046243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.046517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.046780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] PATH_INFO: `/v3/projects/cccb1df487d7474d87189e88b6b52993/users/088b4dce25a049f0b734e7f2035f90c9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.062429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccb1df487d7474d87189e88b6b52993, user_id=088b4dce25a049f0b734e7f2035f90c9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.065124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.080184 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1242/2445] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:55:59 2022] PUT /identity/v3/projects/cccb1df487d7474d87189e88b6b52993/users/088b4dce25a049f0b734e7f2035f90c9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:59.085247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.097219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1031-aa64-4658-b955-c745f8448c48 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.163282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.163889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.164127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.164400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.165065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.166657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fea8334-caaf-441c-92e8-8fb3a8b8b5c5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.176662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1204/2446] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:55:59.181181 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.192332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff9bb93-d77d-4422-be87-600abb134fed admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.197483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.198137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.198411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.198677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] PATH_INFO: `/v3/projects/cccb1df487d7474d87189e88b6b52993/users/088b4dce25a049f0b734e7f2035f90c9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.200711 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccb1df487d7474d87189e88b6b52993, user_id=088b4dce25a049f0b734e7f2035f90c9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.203369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.214290 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1243/2447] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:55:59 2022] PUT /identity/v3/projects/cccb1df487d7474d87189e88b6b52993/users/088b4dce25a049f0b734e7f2035f90c9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:55:59.224908 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.225225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.225499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.253153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] MFA Rules not processed for user `088b4dce25a049f0b734e7f2035f90c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:59.254363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:59.272353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:59.323176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1205/2448] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:59.336063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.338009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:59.362147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:59.400218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49508-95fb-442a-b6c7-622edb0735e2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.404214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.404828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.405140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.405411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.416043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.417544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.438706 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1244/2449] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:59.526203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:59.535842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6507335-3977-4ef3-ad84-8e0b7e7ef2f0 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:59.540009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:59.540622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:59.540933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:59.541191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:59.606712 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:59.608153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:59.617756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1206/2450] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:08.681100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:08.682780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:08.798611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87322810-36ef-4eb1-8341-589aa0856e45 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:08.805235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:08.806109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:08.807020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:08.807453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:08.809728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:08.826588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:08.829105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4bf06-dd77-4a79-9007-04892679508c service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:08.858272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1245/2451] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:09.012798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:09.097674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78825dba-b0d8-4334-a132-ca5b30663a0d service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:09.102290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:09.102933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:09.103210 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:09.103511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] PATH_INFO: `/v3/users/e09ca7517c844f2ea7b4541b6e902037` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:09.104632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e09ca7517c844f2ea7b4541b6e902037)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:09.106725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:09.158990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e09ca7517c844f2ea7b4541b6e902037'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.170593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e09ca7517c844f2ea7b4541b6e902037'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.171345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e09ca7517c844f2ea7b4541b6e902037'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.171959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e09ca7517c844f2ea7b4541b6e902037'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.174438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:09.193515 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e09ca7517c844f2ea7b4541b6e902037'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.211864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1207/2452] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:09 2022] DELETE /identity/v3/users/e09ca7517c844f2ea7b4541b6e902037 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:09.213106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:09.213396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:09.213668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:09.233683 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:09.234915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:09.302643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1246/2453] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:09.313427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:09.317689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:09.392328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd9eb0-5fc0-4c79-8c5f-35480342ef98 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:09.397215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:09.398468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:09.398775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:09.399017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:09.410048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:09.411536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:09.423580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1208/2454] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:09.735449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:09.737182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:09.764266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eef3fa-f196-4c86-8ee8-17371494a8b9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:09.770911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:09.771701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:09.772025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:09.772386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] PATH_INFO: `/v3/projects/d9249d632ba24268ab9c2f5140c6ce25` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:09.774113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9249d632ba24268ab9c2f5140c6ce25)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:09.776191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:09.839109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:09.843186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9249d632ba24268ab9c2f5140c6ce25', 'request_id': 'req-1e76f46e-93ea-4535-b604-fdd296d3d58f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.850851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9249d632ba24268ab9c2f5140c6ce25', 'request_id': 'req-1e76f46e-93ea-4535-b604-fdd296d3d58f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.859289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9249d632ba24268ab9c2f5140c6ce25', 'request_id': 'req-1e76f46e-93ea-4535-b604-fdd296d3d58f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.870412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] The token cache is being invalidate because project d9249d632ba24268ab9c2f5140c6ce25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:09.870718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.871357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:09.872904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1247/2455] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:09 2022] DELETE /identity/v3/projects/d9249d632ba24268ab9c2f5140c6ce25 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:09.877835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:09.888850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb66ae2-e7d1-4ed5-837c-ae3e691b380f service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:09.953312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:09.954072 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:09.954424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:09.955598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] PATH_INFO: `/v3/users/088b4dce25a049f0b734e7f2035f90c9` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:09.959420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=088b4dce25a049f0b734e7f2035f90c9)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:09.961485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:10.015225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088b4dce25a049f0b734e7f2035f90c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.027129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088b4dce25a049f0b734e7f2035f90c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.027129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088b4dce25a049f0b734e7f2035f90c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.028090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088b4dce25a049f0b734e7f2035f90c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.043304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d651345c-c948-4dc6-9368-c8536df84daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088b4dce25a049f0b734e7f2035f90c9'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.049921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:10.060255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1209/2456] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:09 2022] DELETE /identity/v3/users/088b4dce25a049f0b734e7f2035f90c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:10.475867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:10.477626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:10.491195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76f46e-93ea-4535-b604-fdd296d3d58f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:10.566425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:10.567953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.568340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.568716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] PATH_INFO: `/v3/projects/cccb1df487d7474d87189e88b6b52993` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.570638 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccb1df487d7474d87189e88b6b52993)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:10.573762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:10.638853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccb1df487d7474d87189e88b6b52993', 'request_id': 'req-6e328e32-1dbe-40af-ab32-f3c8103a5425'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.646646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccb1df487d7474d87189e88b6b52993', 'request_id': 'req-6e328e32-1dbe-40af-ab32-f3c8103a5425'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.656727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccb1df487d7474d87189e88b6b52993', 'request_id': 'req-6e328e32-1dbe-40af-ab32-f3c8103a5425'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.673944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] The token cache is being invalidate because project cccb1df487d7474d87189e88b6b52993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:10.674440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.674931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6e328e32-1dbe-40af-ab32-f3c8103a5425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:10.677037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1248/2457] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:10 2022] DELETE /identity/v3/projects/cccb1df487d7474d87189e88b6b52993 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:10.682068 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.682354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.682632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.705593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:10.706943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:10.792948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1210/2458] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:10.803391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768408-6c12-482e-a266-1cb8f184467b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.804014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768408-6c12-482e-a266-1cb8f184467b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.804118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768408-6c12-482e-a266-1cb8f184467b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.827775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-21768408-6c12-482e-a266-1cb8f184467b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:10.829612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-21768408-6c12-482e-a266-1cb8f184467b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:10.840338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1249/2459] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:10.844990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:10.857742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c8ed-6d72-486e-8885-d814dc57aa09 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:10.865903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:10.866940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.867306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.867585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.869610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:56:10.870067 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:10.872401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:10.886775 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1211/2460] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:10.894001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21768408-6c12-482e-a266-1cb8f184467b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:10.906107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21768408-6c12-482e-a266-1cb8f184467b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:10.911338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:10.912044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:10.912730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:10.913090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:10.913784 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:10.915778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:10.955874 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1250/2461] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:10 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:10.956404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:10.968689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615692ad-63ad-4763-b564-b14c5d3d441d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.042168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.043353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.043353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.043353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.044109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.046238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.072751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1212/2462] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:10 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:11.077164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.089324 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939beb1-5b9c-45e8-b7ff-a834f677a7f3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.094148 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.094777 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.095055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.095393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.096174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.097700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.113518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1251/2463] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:11.118048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.129479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234efbcb-5640-40f2-b8f0-e10569b27866 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.134487 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.135158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.135420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.135722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.136331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.138138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f737c4-ba47-46a0-a944-e9cf5d64aaf1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.153135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1213/2464] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:11.154494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.165530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0fda4-6d03-413a-b5d8-a5cf00626de2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.171671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.172366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.172641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.172948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] PATH_INFO: `/v3/projects/949db3a3f20e45079bebeb3d58339714/users/82b6fb7fd8a24fc688c4812fec338b81/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.188875 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=949db3a3f20e45079bebeb3d58339714, user_id=82b6fb7fd8a24fc688c4812fec338b81, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.191718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.202571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:11.208856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1252/2465] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:11 2022] PUT /identity/v3/projects/949db3a3f20e45079bebeb3d58339714/users/82b6fb7fd8a24fc688c4812fec338b81/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:11.221227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.221514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.221792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.262396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] MFA Rules not processed for user `82b6fb7fd8a24fc688c4812fec338b81`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:11.263637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:11.317139 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:11.337498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1214/2466] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:11.341990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.419533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36082fc4-79b0-4d65-8d10-b92682b904e1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.424626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.425279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.425561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.426015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.426830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.429070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.479132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1253/2467] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:11 2022] POST /identity/v3/projects => generated 367 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:11.483349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.494931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c749e7-bd04-4eb1-bebb-234ff010de74 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.575851 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.576501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.576779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.577079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.577765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.579784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.616517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1215/2468] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:11 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:11.617964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.631024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418563-1674-4f77-9d86-563823ef8fde admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.636657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.638035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.638035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.638321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.638920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.640602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.657635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1254/2469] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:11.659751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.672532 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4349d-6ca4-4d91-9d02-1d906b8b7e99 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.677863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.678558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.678837 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.679113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] PATH_INFO: `/v3/projects/1d60b06631cc44ddb707115a56ee5a69/users/d225280cb5fc4814a6105b71d94992ca/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.700023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d60b06631cc44ddb707115a56ee5a69, user_id=d225280cb5fc4814a6105b71d94992ca, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.702933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.725133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1216/2470] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:11 2022] PUT /identity/v3/projects/1d60b06631cc44ddb707115a56ee5a69/users/d225280cb5fc4814a6105b71d94992ca/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:11.729485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.745639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afa25ca-6257-4bd5-8e15-77bc502dde93 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.824366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.826078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.826823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.829235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.829499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.831164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f59ae-6a17-40d3-9c05-f4b4cf98cc07 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.846134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1255/2471] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:11.850759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:11.863111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c71c0eb-8869-4abb-b7d3-c88c19839522 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:11.868623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:11.869468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.869748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.870219 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] PATH_INFO: `/v3/projects/1d60b06631cc44ddb707115a56ee5a69/users/d225280cb5fc4814a6105b71d94992ca/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:11.872195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d60b06631cc44ddb707115a56ee5a69, user_id=d225280cb5fc4814a6105b71d94992ca, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:11.894590 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:11.935402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1217/2472] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:11 2022] PUT /identity/v3/projects/1d60b06631cc44ddb707115a56ee5a69/users/d225280cb5fc4814a6105b71d94992ca/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:11.946187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:11.946555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:11.946838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:12.001134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] MFA Rules not processed for user `d225280cb5fc4814a6105b71d94992ca`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:12.002691 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:12.096684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:12.121249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1256/2473] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:12.126975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:12.129399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:12.212949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eee999-0e42-4035-810e-05c6a155fbe8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:12.218648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:12.219353 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:12.226631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:12.226809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:12.238466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:12.239993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20b0c7-46ac-4c8a-bccc-a61a546a7451 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:12.253480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1218/2474] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:56:13.752076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:13.835496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02c3a4-8b90-4453-9893-0ea4f2113467 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:13.842377 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:13.843274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:13.843635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:13.844081 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] PATH_INFO: `/v3/users/82b6fb7fd8a24fc688c4812fec338b81` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:13.845802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82b6fb7fd8a24fc688c4812fec338b81)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:13.847730 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:13.898198 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82b6fb7fd8a24fc688c4812fec338b81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:13.907846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82b6fb7fd8a24fc688c4812fec338b81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:13.908561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82b6fb7fd8a24fc688c4812fec338b81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:13.909369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82b6fb7fd8a24fc688c4812fec338b81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:13.916810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82b6fb7fd8a24fc688c4812fec338b81'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:13.937101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1257/2475] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:13 2022] DELETE /identity/v3/users/82b6fb7fd8a24fc688c4812fec338b81 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:13.942767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:13.942767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:13.942767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:13.962095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:13.963288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:14.006316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:14.025313 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1219/2476] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:56:14.036908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:14.038974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:14.235326 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f3987-c9f4-4e9b-b2e6-e785d4e407f5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:14.239397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:14.239997 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:14.240271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:14.240533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:14.249773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:14.251191 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:14.261207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1258/2477] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:14.580856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:14.582621 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:14.595647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89936e29-28e1-4af2-bbe8-6c7ac5497cdb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:14.601714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:14.602588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:14.608715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:14.608715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] PATH_INFO: `/v3/projects/949db3a3f20e45079bebeb3d58339714` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:14.608715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=949db3a3f20e45079bebeb3d58339714)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:14.608715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:14.657436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '949db3a3f20e45079bebeb3d58339714', 'request_id': 'req-0113e533-c950-4560-a96e-eb4b570cd945'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.665981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '949db3a3f20e45079bebeb3d58339714', 'request_id': 'req-0113e533-c950-4560-a96e-eb4b570cd945'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.677334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '949db3a3f20e45079bebeb3d58339714', 'request_id': 'req-0113e533-c950-4560-a96e-eb4b570cd945'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.696257 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] The token cache is being invalidate because project 949db3a3f20e45079bebeb3d58339714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:14.696539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.697224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.700968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1220/2478] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:14 2022] DELETE /identity/v3/projects/949db3a3f20e45079bebeb3d58339714 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:14.702966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:14.712727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c64eb8-fbf0-4649-a522-50348748ef56 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:14.724812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:14.775522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:14.776185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:14.776442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:14.776871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] PATH_INFO: `/v3/users/d225280cb5fc4814a6105b71d94992ca` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:14.778465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d225280cb5fc4814a6105b71d94992ca)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:14.780374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:14.832737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd225280cb5fc4814a6105b71d94992ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.844157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd225280cb5fc4814a6105b71d94992ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.845079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd225280cb5fc4814a6105b71d94992ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.846060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd225280cb5fc4814a6105b71d94992ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.855457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-ba788482-1f23-4991-8cd3-745faaac3b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd225280cb5fc4814a6105b71d94992ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:14.876829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1259/2479] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:14 2022] DELETE /identity/v3/users/d225280cb5fc4814a6105b71d94992ca => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:15.174830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.176427 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:15.191682 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113e533-c950-4560-a96e-eb4b570cd945 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.256935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.257609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.257867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.258114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] PATH_INFO: `/v3/projects/1d60b06631cc44ddb707115a56ee5a69` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.259404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d60b06631cc44ddb707115a56ee5a69)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.261485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.317228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d60b06631cc44ddb707115a56ee5a69', 'request_id': 'req-71a1895b-4fc1-4daa-aecb-997369230f5c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:15.323615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d60b06631cc44ddb707115a56ee5a69', 'request_id': 'req-71a1895b-4fc1-4daa-aecb-997369230f5c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:15.333972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d60b06631cc44ddb707115a56ee5a69', 'request_id': 'req-71a1895b-4fc1-4daa-aecb-997369230f5c'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:15.341864 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:15.346328 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] The token cache is being invalidate because project 1d60b06631cc44ddb707115a56ee5a69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:15.346761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:15.347687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-71a1895b-4fc1-4daa-aecb-997369230f5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:15.350003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1221/2480] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:15 2022] DELETE /identity/v3/projects/1d60b06631cc44ddb707115a56ee5a69 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:15.355118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.355396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.355663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.375161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:15.376315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:15.443967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1260/2481] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:15.454577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.454803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.455228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.476121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:15.477412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:15.486059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1222/2482] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:15.490674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.501017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346266aa-5d8a-4e19-8fa0-d3d1f8481b35 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.507362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.507969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.508240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.508503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.509182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:56:15.509481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.511069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.525155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1261/2483] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:15.530402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.545358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190688b2-0e8d-4faf-b1c8-7e62ad8e8217 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.552548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.553197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.553480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.553753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.554396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.556306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.590148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1223/2484] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/projects => generated 399 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:15.594884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.609701 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354a72c-a230-417e-9c9c-94c42b0a6d58 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.634308 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:15.671615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.672238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.672512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.672796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.673440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.675442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.700548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1262/2485] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:15.704663 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.716085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469ed8-d837-405f-8655-6566f4450312 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.721242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.721861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.722217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.722491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.723140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.724774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.735580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1224/2486] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:15.739990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.751424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c77bc0-8f76-4592-98f2-ff029ec372dc admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.756782 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.757761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.758212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.758528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.759402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.761032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd7c514-ad84-42c6-808a-829e04a3d818 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.772119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1263/2487] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:15.776442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:15.786726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5428ef-fb6a-4da7-a524-325e0fee463b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:15.792358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:15.793111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.793392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.793668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] PATH_INFO: `/v3/projects/12057d54302b4c09aeb7a394452669de/users/789ab6457c62462f99c6cc88fdfb96d1/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.810653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12057d54302b4c09aeb7a394452669de, user_id=789ab6457c62462f99c6cc88fdfb96d1, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:15.813401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:15.831698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1225/2488] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:15 2022] PUT /identity/v3/projects/12057d54302b4c09aeb7a394452669de/users/789ab6457c62462f99c6cc88fdfb96d1/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:15.844533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:15.845918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:15.846200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:15.897039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] MFA Rules not processed for user `789ab6457c62462f99c6cc88fdfb96d1`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:15.898251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:15.977219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1264/2489] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:15.982065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.066816 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ced65e-630f-42d0-b66a-c36f531f8ee4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.073044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.073719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.074000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.074411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.075403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.077753 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.112235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1226/2490] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:15 2022] POST /identity/v3/projects => generated 401 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:16.115650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.129019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2a73b-7252-4168-a354-b7146ecf8200 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.200090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.200717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.201041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.201284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.202123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.204118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.229903 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1265/2491] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:16 2022] POST /identity/v3/users => generated 307 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:16.231145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.242685 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1337a-1caa-4334-be90-063110a33da4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.249891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.250953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.251548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.252008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.253309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.255696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.268802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1227/2492] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:56:16.273493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.286421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d51da-cfe4-4de7-a7ae-71ec40480b42 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.291877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.292557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.292841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.293145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] PATH_INFO: `/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131/users/e6f3ed14bb9e4a8a856f878a69250203/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.309556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae3e7560d09487aaa99b73bbe0a1131, user_id=e6f3ed14bb9e4a8a856f878a69250203, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.312210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.315831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:16.328271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1266/2493] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:16 2022] PUT /identity/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131/users/e6f3ed14bb9e4a8a856f878a69250203/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:16.332690 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.344105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5174a061-01a3-4a31-abe5-96d0b75696f0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.414140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.414766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.415205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.415483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.416136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.417799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd54521-e404-4bb0-bb84-0b3c05953f2c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.428518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1228/2494] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:16.432826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.444769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c539b8-a63c-46ba-bccc-667e2cddf498 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.449844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.450547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.450827 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.451101 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] PATH_INFO: `/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131/users/e6f3ed14bb9e4a8a856f878a69250203/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.453105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae3e7560d09487aaa99b73bbe0a1131, user_id=e6f3ed14bb9e4a8a856f878a69250203, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.455868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.467736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1267/2495] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:16 2022] PUT /identity/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131/users/e6f3ed14bb9e4a8a856f878a69250203/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:16.478169 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.478557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.478896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.492958 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:16.508073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] MFA Rules not processed for user `e6f3ed14bb9e4a8a856f878a69250203`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:16.509449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:16.587809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1229/2496] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:16.600373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.609114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:16.679452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9455-2aba-4628-afab-6643ad1d89c1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.683353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.684075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.684269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.684596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.695463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.696968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fde13-2ae4-4693-b5da-0fba2956d719 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.707440 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1268/2497] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:16 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:16.790389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:16.800302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bce42-3860-4ee3-baaa-09aca3079d22 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:16.804203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:16.804796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:16.805101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:16.805365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:16.873691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:16.875028 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:16.884013 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1230/2498] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:17.467939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:17.468407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:17.468521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:17.488114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:17.489359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:17.547281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1269/2499] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:17.596326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:17.598024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:17.687435 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a97303-66d9-4493-a73f-d393616ed9b6 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:17.694421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:17.695175 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:17.695479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:17.695764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:17.708327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:17.710060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb82ee-a6e6-41b9-81c9-9537017bfcc7 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:17.719244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1231/2500] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:56:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:56:21.057758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:21.121868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2252133-551b-4988-bc6b-656c88fcc527 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:21.130015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:21.130436 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:21.130751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:21.130964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] PATH_INFO: `/v3/users/789ab6457c62462f99c6cc88fdfb96d1` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:21.132139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=789ab6457c62462f99c6cc88fdfb96d1)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:21.148353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:21.215558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:21.235551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '789ab6457c62462f99c6cc88fdfb96d1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.247445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '789ab6457c62462f99c6cc88fdfb96d1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.254967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '789ab6457c62462f99c6cc88fdfb96d1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.255632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '789ab6457c62462f99c6cc88fdfb96d1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.265847 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '789ab6457c62462f99c6cc88fdfb96d1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.287922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1270/2501] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:21 2022] DELETE /identity/v3/users/789ab6457c62462f99c6cc88fdfb96d1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:21.293356 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:21.293841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:21.294247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:21.328338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:21.333854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:21.355251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:21.413479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1232/2502] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:21.426735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:21.428679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:21.512066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5dd540-73cb-477c-8927-541d7f2f7077 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:21.516972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:21.519868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:21.520686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:21.521194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:21.532617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:21.534524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:21.547271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1271/2503] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:56:21.876297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:21.878464 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:21.892537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e70ce-e1f7-4d5a-962c-4c90e7f780d7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:21.899520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:21.899520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:21.899520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:21.899520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] PATH_INFO: `/v3/projects/12057d54302b4c09aeb7a394452669de` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:21.899926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12057d54302b4c09aeb7a394452669de)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:21.901559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:21.954166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12057d54302b4c09aeb7a394452669de', 'request_id': 'req-369fe1d1-849f-4055-903e-c4a737d90d55'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.964403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12057d54302b4c09aeb7a394452669de', 'request_id': 'req-369fe1d1-849f-4055-903e-c4a737d90d55'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.980919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12057d54302b4c09aeb7a394452669de', 'request_id': 'req-369fe1d1-849f-4055-903e-c4a737d90d55'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.986442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] The token cache is being invalidate because project 12057d54302b4c09aeb7a394452669de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:21.986735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.987388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:21.992006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1233/2504] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:21 2022] DELETE /identity/v3/projects/12057d54302b4c09aeb7a394452669de => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:21.993640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:22.005809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92514d-64b9-49aa-af2e-ac63c4f67f2d service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:22.081627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:22.082983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.083432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.083830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] PATH_INFO: `/v3/users/e6f3ed14bb9e4a8a856f878a69250203` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.085093 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f3ed14bb9e4a8a856f878a69250203)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:22.087236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:22.136033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f3ed14bb9e4a8a856f878a69250203'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.144388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f3ed14bb9e4a8a856f878a69250203'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.145103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f3ed14bb9e4a8a856f878a69250203'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.146041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f3ed14bb9e4a8a856f878a69250203'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.149863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:22.155993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-d082f47c-25e3-448d-94be-9a01f3d63037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f3ed14bb9e4a8a856f878a69250203'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.181224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1272/2505] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:21 2022] DELETE /identity/v3/users/e6f3ed14bb9e4a8a856f878a69250203 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:22.477570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:22.479238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:22.515645 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369fe1d1-849f-4055-903e-c4a737d90d55 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:22.576584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:22.577371 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.577708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.578044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] PATH_INFO: `/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.579407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae3e7560d09487aaa99b73bbe0a1131)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:22.581494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:22.631711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:22.635247 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae3e7560d09487aaa99b73bbe0a1131', 'request_id': 'req-adea4f92-9c84-45ea-874e-41cda39ef123'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.643680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae3e7560d09487aaa99b73bbe0a1131', 'request_id': 'req-adea4f92-9c84-45ea-874e-41cda39ef123'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.652539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae3e7560d09487aaa99b73bbe0a1131', 'request_id': 'req-adea4f92-9c84-45ea-874e-41cda39ef123'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.664441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] The token cache is being invalidate because project 5ae3e7560d09487aaa99b73bbe0a1131 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:22.664728 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.665411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-adea4f92-9c84-45ea-874e-41cda39ef123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:22.666933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1234/2506] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:22 2022] DELETE /identity/v3/projects/5ae3e7560d09487aaa99b73bbe0a1131 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:22.672164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.672327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.672536 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.694273 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:22.695556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:22.771473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1273/2507] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:22.779327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.779635 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.779915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.803951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:22.805241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:22.814672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1235/2508] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:22.819132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:22.829144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c56f1-599f-4cae-a6ed-cf7f74fcc6de None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:22.835289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:22.836065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.836434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.836915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.837657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:56:22.837967 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:22.839654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:22.852456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1274/2509] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:22.856999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:22.870033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f09519-6c4d-4dc3-bd3d-daea63b56ebb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:22.874860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:22.875465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.875743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.876018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.876665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:22.878909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:22.911540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1236/2510] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:22 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:22.917086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:22.928088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f430e-0525-44f6-bca2-3cd0f02ab653 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:22.988346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:22.988878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:22.989178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:22.989445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:22.990068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:22.991976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:23.019291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1275/2511] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:22 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:23.020845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:23.032609 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7da85-1177-48fd-9c06-a73e58547762 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:23.038213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:23.038853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:23.039131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:23.039396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:23.040033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:23.041661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:23.183834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1237/2512] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:23.185135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:23.195864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa25a1-c675-456a-93e0-2bf195a991f6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:23.201067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:23.201704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:23.201980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:23.202247 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:23.202941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:23.204610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d090f-a030-47bd-98dc-49a97d882d28 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:23.217381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1276/2513] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:23.222039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:23.232500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932563db-5f16-4541-ae78-5b2483a51e51 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:23.237823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:23.238619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:23.238831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:23.239120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] PATH_INFO: `/v3/projects/9202ee0ae2084f6f8d846b451f5a9195/users/23cb2ef488974963a34cd8fa0791929b/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:23.257023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9202ee0ae2084f6f8d846b451f5a9195, user_id=23cb2ef488974963a34cd8fa0791929b, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:23.259660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:23.275758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1238/2514] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:23 2022] PUT /identity/v3/projects/9202ee0ae2084f6f8d846b451f5a9195/users/23cb2ef488974963a34cd8fa0791929b/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:23.281114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729dcae-3f57-4d1b-84c4-ecfbef084d9e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:23.281388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729dcae-3f57-4d1b-84c4-ecfbef084d9e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:23.281643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729dcae-3f57-4d1b-84c4-ecfbef084d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:23.309061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f729dcae-3f57-4d1b-84c4-ecfbef084d9e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:23.310291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f729dcae-3f57-4d1b-84c4-ecfbef084d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:23.346115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:23.380502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1277/2515] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:23.391018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:23.393099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:23.454954 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095a64c-9741-40e3-9451-20e1d6568e54 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:23.458916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:23.459538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:23.459846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:23.460108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:23.470615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:23.472009 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:23.483050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1239/2516] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:24.433443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.433949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.434045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.469670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] MFA Rules not processed for user `23cb2ef488974963a34cd8fa0791929b`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:24.471050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:24.563903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.564682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1278/2517] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:24.575614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fe4afd-7884-457d-8f1f-3430813c55f4 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:24.582352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:24.583238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.583518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.583786 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.584424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:24.586402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:24.628317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1240/2518] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:24 2022] POST /identity/v3/projects => generated 369 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:24.632529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.645502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39dc95-3fa3-4763-a19a-12728a8f91cb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:24.717773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:24.718493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.718845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.719218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.719949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:24.722206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:24.754661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1279/2519] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:24 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:24.756123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.778205 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1e806-46d3-40a8-b523-27bc640ef300 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:24.787982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:24.788922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.789791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.789791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.790245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:24.794179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:24.829065 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1241/2520] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:24.833625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.844255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305aeab6-fc9a-43ef-a648-291e74a8c259 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:24.859194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:24.860197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:24.860668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:24.861133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] PATH_INFO: `/v3/projects/ca9d8de04c3c4587a689b69579fa53c6/users/6f9d331cbfca490ca0be524ee3c3087a/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:24.892020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9d8de04c3c4587a689b69579fa53c6, user_id=6f9d331cbfca490ca0be524ee3c3087a, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:24.895736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:24.918509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1280/2521] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:24 2022] PUT /identity/v3/projects/ca9d8de04c3c4587a689b69579fa53c6/users/6f9d331cbfca490ca0be524ee3c3087a/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:24.923523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:24.940151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd46899-5181-48c8-b604-700d1567ca0c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:25.016273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:25.058074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:25.058762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.059131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.059478 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.060243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:25.064797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e6675-12b4-441d-b26c-aeace5265ae7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:25.078906 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1242/2522] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:25.086646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:25.106161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5976cabc-3a16-4785-b0ec-91933b8b051e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:25.118817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:25.119804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.120092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.120443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] PATH_INFO: `/v3/projects/ca9d8de04c3c4587a689b69579fa53c6/users/6f9d331cbfca490ca0be524ee3c3087a/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.123397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca9d8de04c3c4587a689b69579fa53c6, user_id=6f9d331cbfca490ca0be524ee3c3087a, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:25.126313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:25.144513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1281/2523] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:25 2022] PUT /identity/v3/projects/ca9d8de04c3c4587a689b69579fa53c6/users/6f9d331cbfca490ca0be524ee3c3087a/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:25.363149 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.363882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.364439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.399066 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:25.401501 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:25.499221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1243/2524] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:56:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:25.514231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:25.516468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:25.588746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:25.661531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde21fa-3aef-4433-86fc-ce44121c5f28 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:25.667361 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:25.668245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:25.668678 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:25.669145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:25.678680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:25.680630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:25.698330 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1282/2525] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:56:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:56:25.833817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:25.836386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:25.928547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:25.930227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.011602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fed9a-4e51-4f13-b5ba-0de3692a3342 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.029941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.031845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.032387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.032811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.039972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.065958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96748d-9ac8-43db-8f63-7a7df242edac service swift] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.085764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.086839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.087125 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.087449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] PATH_INFO: `/v3/users/771ba65bd2974845ac79d04ad3634c96` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.089284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] RBAC: Authorizing `identity:delete_user(user_id=771ba65bd2974845ac79d04ad3634c96)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.092295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.092479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.099859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa54d8-7aeb-40c0-9046-63ee7d461442 service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.142998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1244/2526] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:26.147629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.149308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.149717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.194671 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771ba65bd2974845ac79d04ad3634c96'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.208145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771ba65bd2974845ac79d04ad3634c96'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.209376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771ba65bd2974845ac79d04ad3634c96'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.212594 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771ba65bd2974845ac79d04ad3634c96'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.217421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] MFA Rules not processed for user `6f9d331cbfca490ca0be524ee3c3087a`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:26.219116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.223878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771ba65bd2974845ac79d04ad3634c96'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.297150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1283/2527] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:25 2022] DELETE /identity/v3/users/771ba65bd2974845ac79d04ad3634c96 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:26.337576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:26.356506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1245/2528] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:26.374319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:26.376410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.440722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:26.494205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ee6c3-b50f-4458-a6bf-3cfc53f7beee admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.502018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.503170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.503459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.503853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.506215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.616199 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:26.617735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.673061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.675382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.694413 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1284/2529] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:26.702057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:26.705201 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.801027 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5c7fa-04a3-4481-b226-32062b76a6a9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.849462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9765f2-60e4-4c34-bcbe-06332c129406 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:26.854604 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.855654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.855935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.856566 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] PATH_INFO: `/v3/projects/845d6ae8d6434c4c96e24cf39f309d23` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.857935 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=845d6ae8d6434c4c96e24cf39f309d23)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.860581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.929259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:26.931964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:26.932935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:26.933661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:26.939192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:26.947064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '845d6ae8d6434c4c96e24cf39f309d23', 'request_id': 'req-62074ead-de86-4f1a-8a95-88925159a384'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.956275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '845d6ae8d6434c4c96e24cf39f309d23', 'request_id': 'req-62074ead-de86-4f1a-8a95-88925159a384'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.964348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:26.966390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:26.971065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '845d6ae8d6434c4c96e24cf39f309d23', 'request_id': 'req-62074ead-de86-4f1a-8a95-88925159a384'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.981526 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1246/2530] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:26.988051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] The token cache is being invalidate because project 845d6ae8d6434c4c96e24cf39f309d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:26.988380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.989203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:26.994150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1285/2531] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:26 2022] DELETE /identity/v3/projects/845d6ae8d6434c4c96e24cf39f309d23 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:26.995849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:27.012060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07101da2-1505-4aaf-92a0-b854005d9b50 service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:27.115744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:27.116783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:27.116929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:27.117832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] PATH_INFO: `/v3/users/9b52efa07ab34e9aae801b2fb2f14ad0` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:27.119239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b52efa07ab34e9aae801b2fb2f14ad0)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:27.121425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:27.188332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:27.239281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b52efa07ab34e9aae801b2fb2f14ad0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.251388 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b52efa07ab34e9aae801b2fb2f14ad0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.254922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b52efa07ab34e9aae801b2fb2f14ad0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.259917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b52efa07ab34e9aae801b2fb2f14ad0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.276827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b52efa07ab34e9aae801b2fb2f14ad0'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.293878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1247/2532] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:26 2022] DELETE /identity/v3/users/9b52efa07ab34e9aae801b2fb2f14ad0 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:27.504625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:27.506157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:27.544483 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62074ead-de86-4f1a-8a95-88925159a384 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:27.557082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:27.619783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:27.620503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:27.620850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:27.621218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] PATH_INFO: `/v3/projects/042e05994c7a4667a52c35869ddaf68a` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:27.622454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042e05994c7a4667a52c35869ddaf68a)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:27.624485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:27.689959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042e05994c7a4667a52c35869ddaf68a', 'request_id': 'req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.698576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042e05994c7a4667a52c35869ddaf68a', 'request_id': 'req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.709207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042e05994c7a4667a52c35869ddaf68a', 'request_id': 'req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.724364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] The token cache is being invalidate because project 042e05994c7a4667a52c35869ddaf68a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:27.724677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.725434 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4762d93c-b2a9-4f3e-84b5-937349b8dcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:27.727900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1286/2533] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:27 2022] DELETE /identity/v3/projects/042e05994c7a4667a52c35869ddaf68a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:28.302239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:28.303752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:28.373662 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560b747-3acd-4a1c-a531-b75a80a9bf7a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:28.435052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:28.435586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:28.436005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:28.436282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:28.438486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:28.514678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:28.517204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6449f4-a846-4c6a-8795-2630848d2c64 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:28.531363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1248/2534] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:37.719966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:38.526536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:39.798788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:39.799357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:39.799357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:39.801087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1287/2535] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:39.809225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9dde-ed14-4a76-84dd-9fe075b5bfcc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:39.809861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9dde-ed14-4a76-84dd-9fe075b5bfcc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:39.810200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82f9dde-ed14-4a76-84dd-9fe075b5bfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:39.834599 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b82f9dde-ed14-4a76-84dd-9fe075b5bfcc None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:39.836150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b82f9dde-ed14-4a76-84dd-9fe075b5bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:39.927138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1249/2536] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:39.938642 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:39.947384 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:40.007629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52010c31-2b88-4bcb-b11d-5c8f7f5c03aa None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:40.012202 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:40.012878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.013178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.013430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.035895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:40.037426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b174539-93fc-48bc-91d4-d06a27b06450 service glance] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:40.050218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1288/2537] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:40.127422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.127733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.128006 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] PATH_INFO: `/` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.129457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1250/2538] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:40.134461 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6741-c6ab-49e7-b6af-43d9acc25126 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.134744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6741-c6ab-49e7-b6af-43d9acc25126 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.135007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6741-c6ab-49e7-b6af-43d9acc25126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.156302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4bef6741-c6ab-49e7-b6af-43d9acc25126 None None] MFA Rules not processed for user `6ce3f12892d141c88071095f6fe6506d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:40.157530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4bef6741-c6ab-49e7-b6af-43d9acc25126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:40.209000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1289/2539] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:56:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:40.221170 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:40.223063 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:40.239421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d67b5-a13d-46d5-bfe2-d9f42a353100 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:40.243799 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:40.244415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.244719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.245045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.252667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:40.254362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba0d4b-39ed-4374-bb05-32fc44eeec3c service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:40.262964 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1251/2540] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:40.352130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125c839-2ea4-40c3-85e0-b33c6c98a040 None None] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.352397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125c839-2ea4-40c3-85e0-b33c6c98a040 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.352648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125c839-2ea4-40c3-85e0-b33c6c98a040 None None] PATH_INFO: `/` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.354109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1290/2541] 10.222.0.18 () {54 vars in 921 bytes} [Tue Sep 13 19:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:56:40.360615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.360981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.361266 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.384711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] MFA Rules not processed for user `844040a2fb60437a9188ccb5293d404b`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:40.385977 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:40.397265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1252/2542] 10.222.0.18 () {58 vars in 1018 bytes} [Tue Sep 13 19:56:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:40.408450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125c839-2ea4-40c3-85e0-b33c6c98a040 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:40.421266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125c839-2ea4-40c3-85e0-b33c6c98a040 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:40.425192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:40.426192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:40.426192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:40.426432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:40.438078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:40.439523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:40.451646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1291/2543] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:42.619407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:42.620867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:42.677741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:42.679793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:42.703875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143134a-dcaf-4b02-b3b1-f9f316562f70 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:42.710100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:42.710720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:42.711003 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:42.711272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:42.713239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:42.742675 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:42.745509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7c2e93-aef3-49d1-9973-d0bcd04bb37b service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:42.760993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1253/2544] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:42.779230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed997118-1a2f-48dc-b8ad-89dd9bde3b2e service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:42.787045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:42.788263 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:42.788402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:42.788815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:42.789121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:42.796740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:42.797051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] PATH_INFO: `/v3/users/23cb2ef488974963a34cd8fa0791929b` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:42.805918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23cb2ef488974963a34cd8fa0791929b)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:42.808121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:42.824858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:42.826329 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:42.859139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:42.864306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23cb2ef488974963a34cd8fa0791929b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:42.873722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23cb2ef488974963a34cd8fa0791929b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:42.874346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23cb2ef488974963a34cd8fa0791929b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:42.875257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23cb2ef488974963a34cd8fa0791929b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:42.882418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23cb2ef488974963a34cd8fa0791929b'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:42.888932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1254/2545] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 19:56:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:42.901917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:42.903300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:42.905338 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1292/2546] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:42 2022] DELETE /identity/v3/users/23cb2ef488974963a34cd8fa0791929b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:42.916960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:42.979811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e5a4b-baa5-4da7-8f29-bdb8a5c9adcf None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:43.041250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:43.041871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:43.042143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:43.042407 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:43.079568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:43.081102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:43.108344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:43.110434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:43.119393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1255/2547] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:56:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:56:43.155318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37e34a-fb9f-43f9-a752-728eda7acee6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:43.160477 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:43.161164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:43.161482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:43.161746 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] PATH_INFO: `/v3/projects/9202ee0ae2084f6f8d846b451f5a9195` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:43.163420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=9202ee0ae2084f6f8d846b451f5a9195)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:43.166047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:43.251520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9202ee0ae2084f6f8d846b451f5a9195', 'request_id': 'req-f64c91c7-d871-413d-b3d7-52d25deca7ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.269617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9202ee0ae2084f6f8d846b451f5a9195', 'request_id': 'req-f64c91c7-d871-413d-b3d7-52d25deca7ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.282757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9202ee0ae2084f6f8d846b451f5a9195', 'request_id': 'req-f64c91c7-d871-413d-b3d7-52d25deca7ca'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.306906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] The token cache is being invalidate because project 9202ee0ae2084f6f8d846b451f5a9195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:43.307269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.308674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.315087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1293/2548] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:43 2022] DELETE /identity/v3/projects/9202ee0ae2084f6f8d846b451f5a9195 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:43.317147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:43.329132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c0a49-1b26-4cc8-a5f6-17a02fe41807 service swift] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:43.412735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:43.434813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:43.435598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:43.435968 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:43.436416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] PATH_INFO: `/v3/users/6f9d331cbfca490ca0be524ee3c3087a` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:43.438397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f9d331cbfca490ca0be524ee3c3087a)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:43.440618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:43.513112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f9d331cbfca490ca0be524ee3c3087a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.523577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f9d331cbfca490ca0be524ee3c3087a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.524270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f9d331cbfca490ca0be524ee3c3087a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.525168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f9d331cbfca490ca0be524ee3c3087a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.555540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-46c8afa3-1403-4ca5-b76c-9a3419444b27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f9d331cbfca490ca0be524ee3c3087a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:43.592673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1256/2549] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:43 2022] DELETE /identity/v3/users/6f9d331cbfca490ca0be524ee3c3087a => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:43.919969 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:43.921493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:43.954275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c91c7-d871-413d-b3d7-52d25deca7ca admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.050723 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:44.077329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.077983 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.078258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.078520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] PATH_INFO: `/v3/projects/ca9d8de04c3c4587a689b69579fa53c6` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.086565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca9d8de04c3c4587a689b69579fa53c6)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.089354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.154591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca9d8de04c3c4587a689b69579fa53c6', 'request_id': 'req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:44.170270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca9d8de04c3c4587a689b69579fa53c6', 'request_id': 'req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:44.182737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca9d8de04c3c4587a689b69579fa53c6', 'request_id': 'req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:44.197164 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] The token cache is being invalidate because project ca9d8de04c3c4587a689b69579fa53c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:44.197635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:44.198608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b57f4d7e-f07e-48c4-b53a-dfe9ecc37467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:44.201251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1294/2550] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:43 2022] DELETE /identity/v3/projects/ca9d8de04c3c4587a689b69579fa53c6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:44.206699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.207218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.207312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.237673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:44.239794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:44.325697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1257/2551] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:44.333398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.333738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.334068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.354974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:44.356185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:44.364194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1295/2552] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:44.368574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.378862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b308f-5ffb-4a6b-816e-e37ad9c6df52 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.383672 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.384288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.384561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.384828 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.385589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:56:44.385814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.387626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.404670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1258/2553] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:44.405916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.416055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f902af-17e9-4c82-8c3e-c7b9242bbc83 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.420589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.421230 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.421501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.421766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.422415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.424294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.457287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1296/2554] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:56:44.458417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.470172 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9d788-a3a4-45da-9533-4a8c4271c12f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.529256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:44.537494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.538853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.539432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.539898 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.540821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.543790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.569268 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1259/2555] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:44.574553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.585855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08247a6-0e49-4661-b9b8-50594c924398 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.590826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.591681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.592126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.592480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.593237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.595307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.610371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1297/2556] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:44.613424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.623856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b7753-90b8-485e-9373-011823ddde3d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.628771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.629445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.629719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.629993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.630668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.632302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18dd691-db1d-4e76-9956-343458a30f3a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.648949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1260/2557] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:44.654529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.667025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883f711-0e08-40e7-88d1-3e3c2f89aa2f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.672364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.673061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.673337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.673635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] PATH_INFO: `/v3/projects/25e9695f0cf247399b1f236b9b74a842/users/e266b85308df4ca588660ad9241ea6b7/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.691562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25e9695f0cf247399b1f236b9b74a842, user_id=e266b85308df4ca588660ad9241ea6b7, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.694980 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.717046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1298/2558] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:44 2022] PUT /identity/v3/projects/25e9695f0cf247399b1f236b9b74a842/users/e266b85308df4ca588660ad9241ea6b7/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:44.726131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.726661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.726934 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.755543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] MFA Rules not processed for user `e266b85308df4ca588660ad9241ea6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:44.756842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:44.860928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1261/2559] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:44.863015 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.930593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6997eec6-1ccc-4774-afca-098d08447aa9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:44.937180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:44.938200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:44.938734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:44.939142 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:44.940130 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:44.942263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:44.968791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:44.977508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1299/2560] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:44.981519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:44.992467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4db00-eb3b-4383-b9f5-f0e6572a3787 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.051869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.052762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.053203 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.053639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.054451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.056542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.082798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1262/2561] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:44 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:45.086975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:45.098099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9f7ca5-7c46-490f-aa75-963de409b9b2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.102596 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.103321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.103598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.103860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.104492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.106140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.116571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1300/2562] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:45.121024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:45.131386 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e79e7-d86f-4979-baef-d7088130d9c3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.139331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.140415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.141161 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.141431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] PATH_INFO: `/v3/projects/80adef9a6eb3454baf942e5701b04a02/users/45a5dfd3f29049f7967702a24cc93070/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.167296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=80adef9a6eb3454baf942e5701b04a02, user_id=45a5dfd3f29049f7967702a24cc93070, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.170228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.192024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1263/2563] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:45 2022] PUT /identity/v3/projects/80adef9a6eb3454baf942e5701b04a02/users/45a5dfd3f29049f7967702a24cc93070/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:45.193231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:45.204025 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a9c1f-1c09-4cbf-bb60-b9f2e747300e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.263095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.263896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.264276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.264650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.265560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.267270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab522ca-da44-432c-a799-e8c6f218e903 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.279429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1301/2564] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:45.283631 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:45.296336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918335ad-2a42-46d2-a8cc-cf49e7e499cc admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.301254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.301916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.302189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.302456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] PATH_INFO: `/v3/projects/80adef9a6eb3454baf942e5701b04a02/users/45a5dfd3f29049f7967702a24cc93070/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.304341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80adef9a6eb3454baf942e5701b04a02, user_id=45a5dfd3f29049f7967702a24cc93070, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.307033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.317450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1264/2565] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:45 2022] PUT /identity/v3/projects/80adef9a6eb3454baf942e5701b04a02/users/45a5dfd3f29049f7967702a24cc93070/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:45.387805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.388033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.388266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.419939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] MFA Rules not processed for user `45a5dfd3f29049f7967702a24cc93070`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:45.421301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:45.496427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1302/2566] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:45.507567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:45.509484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:45.534936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:45.574460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88bc06c-be4d-46e0-8e2b-6bd270a60793 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:45.578204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:45.578810 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:45.579081 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:45.579348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:45.659838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:45.661447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10154b-b53b-41a5-b401-f73190333708 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:45.678265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1265/2567] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:48.329905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:48.402523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030a371a-2b7a-4eb4-82e3-7ea7933cb3cb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:48.407618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:48.408274 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:48.408558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:48.408849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] PATH_INFO: `/v3/users/e266b85308df4ca588660ad9241ea6b7` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:48.409992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e266b85308df4ca588660ad9241ea6b7)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:48.411909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:48.425454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:48.472469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e266b85308df4ca588660ad9241ea6b7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:48.482226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e266b85308df4ca588660ad9241ea6b7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:48.483152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e266b85308df4ca588660ad9241ea6b7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:48.483920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e266b85308df4ca588660ad9241ea6b7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:48.491920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e266b85308df4ca588660ad9241ea6b7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:48.509868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1303/2568] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:48 2022] DELETE /identity/v3/users/e266b85308df4ca588660ad9241ea6b7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:48.515127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:48.517880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:48.518463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:48.568089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:48.569368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:48.654727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1266/2569] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:48.665737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:48.667447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:48.735886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e3aa5-df1b-40d4-8c25-3eaebc5a2484 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:48.739579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:48.740207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:48.740484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:48.740755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:48.750776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:48.752219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:48.764075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1304/2570] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:49.107331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:49.108800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:49.123430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51ceb-46b1-4af1-a555-137b6bb15b28 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:49.134367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:49.135060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:49.135403 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:49.135670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] PATH_INFO: `/v3/projects/25e9695f0cf247399b1f236b9b74a842` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:49.137282 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] RBAC: Authorizing `identity:delete_project(project_id=25e9695f0cf247399b1f236b9b74a842)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:49.139225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:49.213354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25e9695f0cf247399b1f236b9b74a842', 'request_id': 'req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.221739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25e9695f0cf247399b1f236b9b74a842', 'request_id': 'req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.230929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25e9695f0cf247399b1f236b9b74a842', 'request_id': 'req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.236503 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:49.245834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] The token cache is being invalidate because project 25e9695f0cf247399b1f236b9b74a842 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:49.246090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.246765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.248861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1267/2571] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:49 2022] DELETE /identity/v3/projects/25e9695f0cf247399b1f236b9b74a842 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:49.258382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:49.270940 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c637b4-a24c-423b-8ac1-4f6065290c69 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:49.351123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:49.351822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:49.352006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:49.352271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] PATH_INFO: `/v3/users/45a5dfd3f29049f7967702a24cc93070` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:49.353511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a5dfd3f29049f7967702a24cc93070)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:49.355408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:49.410018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:49.422798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a5dfd3f29049f7967702a24cc93070'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.431113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a5dfd3f29049f7967702a24cc93070'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.431881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a5dfd3f29049f7967702a24cc93070'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.432581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a5dfd3f29049f7967702a24cc93070'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.439681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c37b4c6f-4882-417e-99fd-0399cc836b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a5dfd3f29049f7967702a24cc93070'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:49.480345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1305/2572] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:56:49 2022] DELETE /identity/v3/users/45a5dfd3f29049f7967702a24cc93070 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:49.938124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:49.939315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:49.963891 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c79eeb4-b2e5-44bd-afcf-3d79adb2177f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.040739 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.041418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.041692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.041959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] PATH_INFO: `/v3/projects/80adef9a6eb3454baf942e5701b04a02` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.043121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80adef9a6eb3454baf942e5701b04a02)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.045079 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.098618 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80adef9a6eb3454baf942e5701b04a02', 'request_id': 'req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:50.105233 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80adef9a6eb3454baf942e5701b04a02', 'request_id': 'req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:50.114443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80adef9a6eb3454baf942e5701b04a02', 'request_id': 'req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:50.126794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] The token cache is being invalidate because project 80adef9a6eb3454baf942e5701b04a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:56:50.127464 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:50.128114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-117e9e89-5d58-4d86-aeb7-f6e3d16a3316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:56:50.130298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1268/2573] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:56:49 2022] DELETE /identity/v3/projects/80adef9a6eb3454baf942e5701b04a02 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:56:50.135173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.135532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.135866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.158896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:50.160097 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:50.234537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1306/2574] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:56:50.242275 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.242499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.242747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.263336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:50.264535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:50.272657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1269/2575] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:56:50.276922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.288590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811390b8-4807-4f05-977d-eb47baee8858 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.293756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.294584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.295020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.295515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.296380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:56:50.296848 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.298956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.312613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1307/2576] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:56:50.317244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.326537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a69377-2ab8-4be7-bb21-1f0ade6cc3c8 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.331278 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.331875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.332171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.332442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.333096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.334995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.369656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1270/2577] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:56:50 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:56:50.370735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.382190 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a131d0-161c-4061-a2a8-57cda917c3b0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.458897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.459524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.459801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.460069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.460703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.462702 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.492231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1308/2578] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:56:50 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:56:50.493619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.505962 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89598b1-8d71-4d61-8942-b7c51964411b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.510838 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.511480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.511760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.512048 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.512687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.514338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.528001 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1271/2579] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:50.529267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.540267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe89eaa3-ce47-4cc9-86f9-f0baeec0184b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.545217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.545931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.546276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.546629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.547355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.549115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ba995-c31e-4265-9a52-4b64fa60f6d6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.560695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1309/2580] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:56:50.565007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.577633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86638d39-969c-46f9-8e58-7c8f80d58d3f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.586916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.586916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.586916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.586916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] PATH_INFO: `/v3/projects/0237947140534962a00100bc5c1a1ae8/users/c4fd974596744d70bd9c36ed39b9b127/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.609061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0237947140534962a00100bc5c1a1ae8, user_id=c4fd974596744d70bd9c36ed39b9b127, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.613100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.629888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1272/2581] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:56:50 2022] PUT /identity/v3/projects/0237947140534962a00100bc5c1a1ae8/users/c4fd974596744d70bd9c36ed39b9b127/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:56:50.642837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470580-98ef-468b-86f0-37c3424302f9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.643317 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470580-98ef-468b-86f0-37c3424302f9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.643956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78470580-98ef-468b-86f0-37c3424302f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.677003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-78470580-98ef-468b-86f0-37c3424302f9 None None] MFA Rules not processed for user `c4fd974596744d70bd9c36ed39b9b127`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:50.678266 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-78470580-98ef-468b-86f0-37c3424302f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:50.683892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:50.759177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1310/2582] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:50.772112 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:50.774317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:50.803482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:50.841110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9462d3-24d9-4f87-b872-0a6a29e2182f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:50.849391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2771f4c99b6d4cf3a23a5cdb7af09b7c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:50.849391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:50.849391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:50.849391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:50.862302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:50.863754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:50.874017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1273/2583] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:56.493132 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:56.493722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:56.493722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:56.514438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:56:56.515645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:56.580733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1311/2584] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:56:56.593736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:56.595334 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:56.672754 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1d8bbe-c8a0-4f8b-841c-8de2138e3d42 service glance] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:56.677978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:56.678596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:56.678867 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:56.679131 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:56.688231 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:56.689936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:56.698143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1274/2585] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 19:56:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:57:06.568661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=49 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:06.695070 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=48 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:15.063328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:15.064912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:15.122286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12ccc3-faae-4a0b-8063-fb0fe3317e9f None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:15.127880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:15.128814 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:15.129217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:15.129612 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:15.132321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:15.200810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:15.202339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e63e41-b03d-4f7a-b8a5-f61b8ff7fc56 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:15.212727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1312/2586] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:16.829911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:16.841928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d3878-112e-4a52-8f2b-13249cd6f3f5 service swift] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:16.846793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:16.848940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:16.848940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:16.848940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] PATH_INFO: `/v3/users/c4fd974596744d70bd9c36ed39b9b127` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:16.850317 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4fd974596744d70bd9c36ed39b9b127)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:16.852479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:16.914451 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4fd974596744d70bd9c36ed39b9b127'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:16.923727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4fd974596744d70bd9c36ed39b9b127'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:16.924698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4fd974596744d70bd9c36ed39b9b127'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:16.925516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4fd974596744d70bd9c36ed39b9b127'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:16.946015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4fd974596744d70bd9c36ed39b9b127'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:16.960404 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1275/2587] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:16 2022] DELETE /identity/v3/users/c4fd974596744d70bd9c36ed39b9b127 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:16.964992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:16.965559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:16.965559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:16.985520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:16.986768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:17.072227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1313/2588] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:17.079304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:17.080820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:17.150707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc30bc6-4f81-41df-8233-07168825a89e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:17.155248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:17.155974 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.156418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.156630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.167877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:17.169653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90203ee5-9695-47b0-be82-8abc96749021 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:17.190528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1276/2589] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:17.507225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:17.509123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:17.524171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c960fd3-af89-405f-918e-2352374fe3c8 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:17.532438 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:17.533336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.533733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.534224 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] PATH_INFO: `/v3/projects/0237947140534962a00100bc5c1a1ae8` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.535599 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0237947140534962a00100bc5c1a1ae8)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:17.539063 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:17.604824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0237947140534962a00100bc5c1a1ae8', 'request_id': 'req-796de950-2413-4b0f-95d9-498f5f17df2d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:17.612021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:17.618631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0237947140534962a00100bc5c1a1ae8', 'request_id': 'req-796de950-2413-4b0f-95d9-498f5f17df2d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:17.634752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0237947140534962a00100bc5c1a1ae8', 'request_id': 'req-796de950-2413-4b0f-95d9-498f5f17df2d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:17.658508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] The token cache is being invalidate because project 0237947140534962a00100bc5c1a1ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:17.658797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:17.659514 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-796de950-2413-4b0f-95d9-498f5f17df2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:17.661215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1314/2590] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:17 2022] DELETE /identity/v3/projects/0237947140534962a00100bc5c1a1ae8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:17.666567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.666880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.667121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.687692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:17.688909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:17.756719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1277/2591] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:17.764456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.764754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.765061 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.786677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:17.787899 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:17.797112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1315/2592] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:57:17.801436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:17.811751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71dc07-b6a7-4bec-83bf-27caed31c7bc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:17.816498 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:17.817138 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.817413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.817676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.818326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:17.818622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:17.820213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:17.832921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1278/2593] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:17.837503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:17.847906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b48431-b352-4230-9bc3-4c86cf9fb7e1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:17.853391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:17.854439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.854722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.854993 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.855680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:17.857934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:17.889226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1316/2594] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:17 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:17.893573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:17.899665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:17.906735 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bda6bd-c4f3-4195-8346-aa860ce3915b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:17.986926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:17.987568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:17.987849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:17.988959 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:17.989608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:17.991576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.035587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1279/2595] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:17 2022] POST /identity/v3/users => generated 290 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:57:18.035917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:18.049999 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc6c45-79dd-4c28-afa5-86bfc0642ca0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:18.055476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:18.056172 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.056450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.056769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.058295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:18.060176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.077411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1317/2596] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:18.081342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:18.091736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41310-0677-4c94-9f35-0496575605b2 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:18.096391 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:18.097056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.097333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.097600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.098238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:18.099841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b0151-3a13-437c-b09f-c2162a2edc25 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.112189 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1280/2597] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:18.117979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:18.137921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ff6e-db57-4473-a87e-933ca7567f8f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:18.144772 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:18.145652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.146136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.146735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] PATH_INFO: `/v3/projects/245af7b7ed61405ba444cddd26fc69fd/users/4f63b1c74b4b4d34b33bef2b6267ecb2/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.166960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=245af7b7ed61405ba444cddd26fc69fd, user_id=4f63b1c74b4b4d34b33bef2b6267ecb2, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:18.169853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.193032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1318/2598] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:18 2022] PUT /identity/v3/projects/245af7b7ed61405ba444cddd26fc69fd/users/4f63b1c74b4b4d34b33bef2b6267ecb2/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:18.198416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.198702 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.198972 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.223269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:18.225348 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:18.315538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1281/2599] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:18.328259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:18.330510 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:18.431781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af666569-a51d-48a7-9068-9601f8273f4e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:18.435480 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:18.436096 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.436369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.436634 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.447067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:18.448573 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.458928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1319/2600] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:18.590943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:18.592602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:18.607644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13edf2b-26e1-4ad3-be57-0a717cb8da69 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:18.612869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:18.613714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:18.614627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:18.614627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] PATH_INFO: `/v3/users/e60df169b5dc4aa59b5b0246544ef2b6` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:18.616144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60df169b5dc4aa59b5b0246544ef2b6)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:18.618316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:18.672467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60df169b5dc4aa59b5b0246544ef2b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:18.680377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60df169b5dc4aa59b5b0246544ef2b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:18.681160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60df169b5dc4aa59b5b0246544ef2b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:18.681833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60df169b5dc4aa59b5b0246544ef2b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:18.698054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-33c7afd0-990d-4933-ae7e-3c48f30e486e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60df169b5dc4aa59b5b0246544ef2b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:18.702832 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:18.713202 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1282/2601] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:18 2022] DELETE /identity/v3/users/e60df169b5dc4aa59b5b0246544ef2b6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:19.006356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.016740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58fa0fd-81a8-406b-b217-8258a7ee9ccd service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.101005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.101766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.101936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.102203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] PATH_INFO: `/v3/projects/3ee86defec834a3c9613ca3dfa5e4350` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.103365 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee86defec834a3c9613ca3dfa5e4350)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.106429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.125698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:19.163587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee86defec834a3c9613ca3dfa5e4350', 'request_id': 'req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:19.179385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee86defec834a3c9613ca3dfa5e4350', 'request_id': 'req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:19.191208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee86defec834a3c9613ca3dfa5e4350', 'request_id': 'req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:19.211150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] The token cache is being invalidate because project 3ee86defec834a3c9613ca3dfa5e4350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:19.211704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:19.213105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-bf9fd8b7-2a44-4ae2-b64a-20c3b6a6ae21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:19.218528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1320/2602] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:19 2022] DELETE /identity/v3/projects/3ee86defec834a3c9613ca3dfa5e4350 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:19.220191 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.220684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.220767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.246228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:19.247492 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:19.335689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1283/2603] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:19.338802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.339090 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.339369 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.363197 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:19.364430 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:19.376932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1321/2604] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:19.379480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.390059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bcee-0cfe-48ff-be07-0be40c1cae97 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.395416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.397038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.397038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.397038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.397352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:19.397567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.399105 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.416835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1284/2605] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:19.418088 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.428481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc4626-37cd-448f-a7f7-f5e166ddc685 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.433381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.433989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.434309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.434728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.435665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.437412 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.482698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1322/2606] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:19 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:19.487423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.503463 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80339a1f-7f8f-4742-bd81-919bd8c7ccf8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.591909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.593140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.593428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.593752 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.594578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.597922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.625283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1285/2607] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:19 2022] POST /identity/v3/users => generated 293 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:19.629276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.649209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795b184-f690-4814-9413-f39b0743d990 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.663379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.664425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.664792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.665902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.666648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.669334 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.691275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1323/2608] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:19.695542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.708743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed6010-0173-4543-856a-1523baae3298 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.715295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.716240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.716614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.717019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.717772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.719495 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cba6c-2474-499c-a30f-351ca81ded2f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.734803 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1286/2609] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:19.738908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.761429 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67bc59-c691-4a31-8de3-e4d60e2d7801 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:19.767079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:19.767769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.768047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.769004 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] PATH_INFO: `/v3/projects/d7256eb917624a56a2762fcdc3d11b29/users/02bb08e3dee645e4abe2e6d5e106ea57/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.793353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7256eb917624a56a2762fcdc3d11b29, user_id=02bb08e3dee645e4abe2e6d5e106ea57, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:19.796026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:19.824294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1324/2610] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:19 2022] PUT /identity/v3/projects/d7256eb917624a56a2762fcdc3d11b29/users/02bb08e3dee645e4abe2e6d5e106ea57/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:57:19.829629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a254-d634-4093-b37b-ef4474932165 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:19.830104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a254-d634-4093-b37b-ef4474932165 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:19.830579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a254-d634-4093-b37b-ef4474932165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:19.856441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-90d4a254-d634-4093-b37b-ef4474932165 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:19.858094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-90d4a254-d634-4093-b37b-ef4474932165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:19.942862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:19.953483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1287/2611] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:19.963561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:19.967057 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:20.046276 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6483b6f3-774d-4adf-b419-1d46db7d907b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:20.051122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:20.051866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:20.052200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:20.052537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:20.066470 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:20.069748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:20.085544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1325/2612] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:24.924291 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:24.924917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:24.924917 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:24.955611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] MFA Rules not processed for user `4f63b1c74b4b4d34b33bef2b6267ecb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:24.957372 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:25.078441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1288/2613] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:25.084398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.086864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:25.110696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a092c04-9d15-4987-a802-9408c02feded service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.116855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.120728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.121169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.121367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.122239 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.124504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.167038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1326/2614] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:25 2022] POST /identity/v3/projects => generated 377 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:25.171093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.195895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28387d1-68e3-4ed5-a274-4d3904d7ef31 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.304218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.304970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.305261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.305557 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.306362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.308683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.340983 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1289/2615] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:25 2022] POST /identity/v3/users => generated 295 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:25.343831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.358155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc3615-2eaf-45bd-9769-90d2b5cf428c admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.363560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.364458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.364737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.365133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.365895 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.368000 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.386781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1327/2616] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:57:25.387693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.401686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789d021-0de3-429f-9e27-619093f1c9af admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.408535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.409790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.410446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.412158 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] PATH_INFO: `/v3/projects/936d970c33ff455983a5656d5e3c88c7/users/0aed6c0f3bea4362a4c78b7f4d0522fe/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.438180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936d970c33ff455983a5656d5e3c88c7, user_id=0aed6c0f3bea4362a4c78b7f4d0522fe, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.441809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.468584 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1290/2617] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:25 2022] PUT /identity/v3/projects/936d970c33ff455983a5656d5e3c88c7/users/0aed6c0f3bea4362a4c78b7f4d0522fe/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:57:25.471235 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.489705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2fbec-24b5-451c-906b-547bfaa2ebd7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.498966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:25.579677 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.580123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.580404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.580916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.581695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.584924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a66e19-d395-41df-9ec5-10e755abeb87 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.602928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1328/2618] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:57:25.604680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:25.621289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58072a5e-1aab-40fc-b7e2-7a94266dbc46 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:25.630733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:25.631576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:25.631926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:25.632300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] PATH_INFO: `/v3/projects/936d970c33ff455983a5656d5e3c88c7/users/0aed6c0f3bea4362a4c78b7f4d0522fe/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:25.635061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=936d970c33ff455983a5656d5e3c88c7, user_id=0aed6c0f3bea4362a4c78b7f4d0522fe, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:25.642413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:25.664470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1291/2619] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:25 2022] PUT /identity/v3/projects/936d970c33ff455983a5656d5e3c88c7/users/0aed6c0f3bea4362a4c78b7f4d0522fe/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:27.191498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469fe03-43f9-4a43-b681-3b409fd2ab03 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:27.192585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469fe03-43f9-4a43-b681-3b409fd2ab03 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:27.193058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469fe03-43f9-4a43-b681-3b409fd2ab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:27.239946 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6469fe03-43f9-4a43-b681-3b409fd2ab03 None None] MFA Rules not processed for user `02bb08e3dee645e4abe2e6d5e106ea57`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:27.241725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6469fe03-43f9-4a43-b681-3b409fd2ab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:27.388072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1329/2620] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:27.421432 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:27.425716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:27.514089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:27.579806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e11b7e-2b44-4cb3-8843-38ee6d28bf2f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:27.594715 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:27.595365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:27.595647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:27.596034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:27.605980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:27.608666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf7aae-39b7-4711-b5ef-abff352a3503 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:27.633496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1292/2621] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:27 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:29.871065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:29.871607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:29.871808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:29.899303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] MFA Rules not processed for user `02bb08e3dee645e4abe2e6d5e106ea57`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:29.900458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:29.923744 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1330/2622] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:32.604050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:32.605312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:32.606383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:32.651882 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] MFA Rules not processed for user `0aed6c0f3bea4362a4c78b7f4d0522fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:32.654658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:32.761096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1293/2623] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:32.779315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:32.781878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:32.808354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90a083-0e90-4e8e-80fc-4b289ef6b1c9 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:32.814989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:32.815800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:32.816083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:32.816350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:32.935123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:32.936798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:32.958973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1331/2624] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:39.381335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:39.382817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:39.450336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7bd72-8711-41be-8690-9745f5a6f077 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:39.454049 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:39.454658 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:39.454924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:39.455186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:39.457061 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:39.532841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:39.534378 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:39.547522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1294/2625] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:42.943865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=88 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:43.260693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:43.270292 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:43.288641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24412fda-fe2a-4ee0-9751-44f522cc8267 service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:43.295104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:43.296003 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.296285 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.296639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] PATH_INFO: `/v3/users/dbea9a6a35c6445a9ab76ee7278abda4` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.298227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbea9a6a35c6445a9ab76ee7278abda4)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:43.300123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:43.357409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbea9a6a35c6445a9ab76ee7278abda4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.367591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbea9a6a35c6445a9ab76ee7278abda4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.368315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbea9a6a35c6445a9ab76ee7278abda4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.369023 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbea9a6a35c6445a9ab76ee7278abda4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.384155 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-78b558b3-2451-4cf1-abc3-edebc5948c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbea9a6a35c6445a9ab76ee7278abda4'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.404931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1332/2626] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:43 2022] DELETE /identity/v3/users/dbea9a6a35c6445a9ab76ee7278abda4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:43.601488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:43.612589 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:43.619564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2271d47b-616c-45dc-a35b-693b7d148edb service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:43.685351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:43.685998 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.686279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.686552 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] PATH_INFO: `/v3/projects/cd32f4615dd0450f8cc175e0bfcfc4a1` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.687734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd32f4615dd0450f8cc175e0bfcfc4a1)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:43.689725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:43.746090 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd32f4615dd0450f8cc175e0bfcfc4a1', 'request_id': 'req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.754160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd32f4615dd0450f8cc175e0bfcfc4a1', 'request_id': 'req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.763680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd32f4615dd0450f8cc175e0bfcfc4a1', 'request_id': 'req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.775836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] The token cache is being invalidate because project cd32f4615dd0450f8cc175e0bfcfc4a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:43.778462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.778462 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-3a55495b-b6b9-42fd-b1bb-c29c3afd4215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:43.781736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1295/2627] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:43 2022] DELETE /identity/v3/projects/cd32f4615dd0450f8cc175e0bfcfc4a1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:43.784040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.784526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.784807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.807099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:43.808400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:43.872716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1333/2628] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:43.880643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.881018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.881423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.907380 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:43.908642 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:43.917834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1296/2629] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:43.922577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:43.933808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbadb69-259c-4213-8c19-04b4eb84344b None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:43.941485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:43.942531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.942894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.943257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.944242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:43.945260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:43.947337 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:43.962194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1334/2630] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:43.967241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:43.977873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c891e-e156-4234-bc89-27440d7f2979 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:43.982989 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:43.983625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:43.983910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:43.984280 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:43.985091 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:43.986900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.018422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1297/2631] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:43 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:57:44.023713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:44.040253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585dfea5-41ce-481e-b5cf-754ed75752c0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:44.110598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:44.111223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.111733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.112325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.113169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:44.117512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.126379 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:44.147524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1335/2632] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:44 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:44.149080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:44.159869 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ec00f-7147-4d90-8adf-132527027734 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:44.165711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:44.165965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.166269 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.166848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.167689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:44.169669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.184015 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1298/2633] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:44.188408 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:44.199410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775b8fc-7191-4631-9401-435fc59008fc admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:44.205204 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:44.205845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.206100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.206359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.206995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:44.208647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69049ca6-fee8-442c-8674-a77b7d72fca5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.227156 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1336/2634] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:44.231507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:44.242084 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370c64b-0a64-42e9-897c-2a0cc88e35a9 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:44.246969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:44.247659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.247937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.248215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] PATH_INFO: `/v3/projects/e2b8b166479947789ba71165567b218b/users/802641ed01f4485fa1b00db9e9885514/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.266195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b8b166479947789ba71165567b218b, user_id=802641ed01f4485fa1b00db9e9885514, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:44.268873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.285661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1299/2635] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:44 2022] PUT /identity/v3/projects/e2b8b166479947789ba71165567b218b/users/802641ed01f4485fa1b00db9e9885514/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:44.296650 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59355d-0c34-4297-b9c4-1bc9f7cb6783 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.297078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59355d-0c34-4297-b9c4-1bc9f7cb6783 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.297439 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59355d-0c34-4297-b9c4-1bc9f7cb6783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.330180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4f59355d-0c34-4297-b9c4-1bc9f7cb6783 None None] MFA Rules not processed for user `802641ed01f4485fa1b00db9e9885514`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:44.331575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4f59355d-0c34-4297-b9c4-1bc9f7cb6783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:44.417837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1337/2636] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:44.429338 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:44.436277 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:44.470195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:44.501854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de068e-5ce9-4309-a53b-c1d1dc4e1c3c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:44.505612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:44.506223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:44.506497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:44.506764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:44.516042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:44.517546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:44.530251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1300/2637] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:50.524457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3a79a-5b58-40f3-87f3-723006464388 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:50.525014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3a79a-5b58-40f3-87f3-723006464388 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:50.525014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3a79a-5b58-40f3-87f3-723006464388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:50.544713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-46e3a79a-5b58-40f3-87f3-723006464388 None None] MFA Rules not processed for user `4f63b1c74b4b4d34b33bef2b6267ecb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:50.545937 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-46e3a79a-5b58-40f3-87f3-723006464388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:50.599964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1338/2638] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:53.111215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:53.183941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aae10fb-8985-467c-8481-a508299079e1 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:53.189785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:53.190724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:53.190924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:53.191135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] PATH_INFO: `/v3/users/802641ed01f4485fa1b00db9e9885514` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:53.192857 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802641ed01f4485fa1b00db9e9885514)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:53.194769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:53.267343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802641ed01f4485fa1b00db9e9885514'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:53.279742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802641ed01f4485fa1b00db9e9885514'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:53.280774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802641ed01f4485fa1b00db9e9885514'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:53.281666 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802641ed01f4485fa1b00db9e9885514'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:53.297430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802641ed01f4485fa1b00db9e9885514'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:53.316923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1301/2639] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:53 2022] DELETE /identity/v3/users/802641ed01f4485fa1b00db9e9885514 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:53.321503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:53.322041 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:53.322269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:53.365027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:53.366465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:53.444589 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1339/2640] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:53.455044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:53.456504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:53.519239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd199f5-2cc6-4824-8295-823758d47a42 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:53.522887 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:53.523490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:53.523766 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:53.524034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:53.530757 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:53.535151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:53.536829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926c268-670c-44c9-8ffb-98a229c1bd01 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:53.549694 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1302/2641] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:54.033716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.037375 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:54.050805 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690a327-f013-4419-84c1-a19f16945ff1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.057424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.058141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.058418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.058681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] PATH_INFO: `/v3/projects/e2b8b166479947789ba71165567b218b` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.059911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b8b166479947789ba71165567b218b)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.061958 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.073643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:54.131690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b8b166479947789ba71165567b218b', 'request_id': 'req-2dbd4a28-5173-4d56-93ee-015d56b64d28'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:54.138475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b8b166479947789ba71165567b218b', 'request_id': 'req-2dbd4a28-5173-4d56-93ee-015d56b64d28'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:54.146543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b8b166479947789ba71165567b218b', 'request_id': 'req-2dbd4a28-5173-4d56-93ee-015d56b64d28'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:54.158725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] The token cache is being invalidate because project e2b8b166479947789ba71165567b218b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:54.159353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:54.160176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-2dbd4a28-5173-4d56-93ee-015d56b64d28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:54.161830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1340/2642] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:54 2022] DELETE /identity/v3/projects/e2b8b166479947789ba71165567b218b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:54.167530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.168127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.168127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.189724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:54.191602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:54.255547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1303/2643] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:54.265570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.265860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.266135 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.288801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:54.290453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:54.298955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1341/2644] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:54.303716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.314814 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55322f8-8017-49a9-a32b-0229d4f32d41 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.321878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.322470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.322740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.323039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.323706 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:54.323985 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.325698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.340298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1304/2645] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:54.344657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.354346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a2fa2-ac68-4e05-b8b4-87b69b8c45f7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.359232 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.360075 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.360349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.360618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.361331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.363287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.394402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1342/2646] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:54.399279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.410430 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cabc4-4983-41dd-82a4-e0c718d66849 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.472824 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.473724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.474043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.474297 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.474961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.476949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.501942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1305/2647] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:54.506316 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.517304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc31150-d4b7-4f73-b454-b048254f076e admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.522843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.523484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.523763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.524079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.524715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.526468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.541398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1343/2648] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:54.542813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.553893 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cf5830-e3bd-4146-9daf-4202f0008733 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.559805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.560656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.560966 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.561240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.561942 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.563627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a81ff4-0dc5-4752-973e-041fb5dfa567 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.576234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1306/2649] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:54.580927 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.592399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5a321-0909-40b1-99e6-ebed2180ce6f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.598463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.599167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.599443 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.599720 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] PATH_INFO: `/v3/projects/49301d45c4df4fa8bc42dfebefe95f88/users/0474b070ff1d44b28ed9b471ce26e9b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.618925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49301d45c4df4fa8bc42dfebefe95f88, user_id=0474b070ff1d44b28ed9b471ce26e9b6, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.621700 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.642418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1344/2650] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:54 2022] PUT /identity/v3/projects/49301d45c4df4fa8bc42dfebefe95f88/users/0474b070ff1d44b28ed9b471ce26e9b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:57:54.654103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.654400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.654629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.686932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] MFA Rules not processed for user `0474b070ff1d44b28ed9b471ce26e9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:54.688167 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:54.740633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:54.762152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1307/2651] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:54.766838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.835982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee8016-4c76-4f54-ace2-947c6ac70064 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.841489 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.842122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.842426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.842673 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.843313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.845616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.875253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1345/2652] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:54.879405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:54.891159 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7410a-c0c5-480c-9e1c-4a81104281c9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:54.958982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:54.959636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:54.959913 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:54.960185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:54.960861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:54.962815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:54.986926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1308/2653] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:54 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:54.991593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.002893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e2650-11a6-4ff8-8f9c-94a93f408ac8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.008200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.009047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.009425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.009803 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.010760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.012560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.036180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1346/2654] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:55.036304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.045283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba6d9c-acce-423a-9a0e-95dbce6d3ee3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.050212 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.050853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.051133 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.051405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] PATH_INFO: `/v3/projects/4978793d1b50491aa0a09128c32d58c7/users/20e9906209b5498480ab56284a6df8c3/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.067037 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4978793d1b50491aa0a09128c32d58c7, user_id=20e9906209b5498480ab56284a6df8c3, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.069729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.084653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1309/2655] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:55 2022] PUT /identity/v3/projects/4978793d1b50491aa0a09128c32d58c7/users/20e9906209b5498480ab56284a6df8c3/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:57:55.089107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.099985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4751979-cb62-4263-bec3-0aa5f85da2ba admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.105217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:55.174037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.174663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.174932 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.175209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.176026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.177662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf63c9d-7631-4107-a255-525a776d9f53 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.188221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1347/2656] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:57:55.192555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.203326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df28a0d7-3c53-4900-8b17-25c13c29c7a8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.208076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.208731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.209053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.209323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] PATH_INFO: `/v3/projects/4978793d1b50491aa0a09128c32d58c7/users/20e9906209b5498480ab56284a6df8c3/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.211264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4978793d1b50491aa0a09128c32d58c7, user_id=20e9906209b5498480ab56284a6df8c3, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.213984 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.225640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1310/2657] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:55 2022] PUT /identity/v3/projects/4978793d1b50491aa0a09128c32d58c7/users/20e9906209b5498480ab56284a6df8c3/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:55.236039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.236341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.236608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.265141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] MFA Rules not processed for user `20e9906209b5498480ab56284a6df8c3`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:55.266347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:55.338989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1348/2658] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:55.350823 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.352854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:55.416173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45f288-a39c-470e-bca2-5cd2aa24c6c2 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.420088 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.420696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.421022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.421276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.431364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.432845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb2ae9-0415-4ec0-99d9-0f1c634d5447 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.442680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1311/2659] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:55.472518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.539001 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edca05-5eb5-41b4-be42-cb65b4465c2c None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.544452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.545223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.545504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.545775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] PATH_INFO: `/v3/users/0474b070ff1d44b28ed9b471ce26e9b6` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.546922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0474b070ff1d44b28ed9b471ce26e9b6)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.549016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.588790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:55.602506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0474b070ff1d44b28ed9b471ce26e9b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:55.612050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0474b070ff1d44b28ed9b471ce26e9b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:55.612670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0474b070ff1d44b28ed9b471ce26e9b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:55.613291 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0474b070ff1d44b28ed9b471ce26e9b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:55.621912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0474b070ff1d44b28ed9b471ce26e9b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:55.640502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1349/2660] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:55 2022] DELETE /identity/v3/users/0474b070ff1d44b28ed9b471ce26e9b6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:55.645225 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.645506 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.645777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.665274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:55.666470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:55.709889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:55.734194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1312/2661] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:55.744279 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:55.745896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:55.805529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e219d-bbf6-4493-af06-e696e9155ae8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:55.809656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:55.810269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:55.810541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:55.810807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:55.820391 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:55.821856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:55.831792 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1350/2662] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:56.187548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:56.189122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:56.201553 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca14f34-df20-46b2-8900-fc853a6b6e9e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:56.206522 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:56.207456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:56.207456 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:56.207724 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] PATH_INFO: `/v3/projects/49301d45c4df4fa8bc42dfebefe95f88` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:56.209107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49301d45c4df4fa8bc42dfebefe95f88)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:56.211176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:56.271187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49301d45c4df4fa8bc42dfebefe95f88', 'request_id': 'req-1218d389-338f-4163-a9ca-21078067ec9d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.277941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49301d45c4df4fa8bc42dfebefe95f88', 'request_id': 'req-1218d389-338f-4163-a9ca-21078067ec9d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.287220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49301d45c4df4fa8bc42dfebefe95f88', 'request_id': 'req-1218d389-338f-4163-a9ca-21078067ec9d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.298509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] The token cache is being invalidate because project 49301d45c4df4fa8bc42dfebefe95f88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:56.298794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.299448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.300957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1313/2663] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:56 2022] DELETE /identity/v3/projects/49301d45c4df4fa8bc42dfebefe95f88 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:56.305062 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:56.315106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ae401-a6cd-44e0-9336-2cfa931c1120 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:56.376914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:56.377619 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:56.377900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:56.378167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] PATH_INFO: `/v3/users/20e9906209b5498480ab56284a6df8c3` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:56.379350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20e9906209b5498480ab56284a6df8c3)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:56.381328 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:56.430002 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20e9906209b5498480ab56284a6df8c3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.434040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:56.438797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20e9906209b5498480ab56284a6df8c3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.439463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20e9906209b5498480ab56284a6df8c3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.440200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20e9906209b5498480ab56284a6df8c3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.449420 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c756bc5-3374-4ced-8a87-667bc98b43b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20e9906209b5498480ab56284a6df8c3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.466632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1351/2664] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:56 2022] DELETE /identity/v3/users/20e9906209b5498480ab56284a6df8c3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:56.754165 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:56.755809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:56.770325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1218d389-338f-4163-a9ca-21078067ec9d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:56.829895 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:56.830448 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:56.830727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:56.831000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] PATH_INFO: `/v3/projects/4978793d1b50491aa0a09128c32d58c7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:56.832196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4978793d1b50491aa0a09128c32d58c7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:56.834109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:56.882839 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4978793d1b50491aa0a09128c32d58c7', 'request_id': 'req-d7a266d5-6a50-4029-b60e-93bd882ad860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.889655 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4978793d1b50491aa0a09128c32d58c7', 'request_id': 'req-d7a266d5-6a50-4029-b60e-93bd882ad860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.898326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4978793d1b50491aa0a09128c32d58c7', 'request_id': 'req-d7a266d5-6a50-4029-b60e-93bd882ad860'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.905042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:56.910409 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] The token cache is being invalidate because project 4978793d1b50491aa0a09128c32d58c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:56.910703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.911668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7a266d5-6a50-4029-b60e-93bd882ad860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:56.913558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1314/2665] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:56 2022] DELETE /identity/v3/projects/4978793d1b50491aa0a09128c32d58c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:56.919251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:56.919776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:56.919776 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:56.939865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:56.941112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:57.004287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1352/2666] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:57.014176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.014434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.014726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.036429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:57.037674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:57.046604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1315/2667] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:57.051200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.061389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d66305-7ff6-4d02-8fa9-4a99da4c0f97 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.066565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.067231 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.067503 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.067764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.068404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:57.068687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.070315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.082973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1353/2668] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:57.087562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.097483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0f3ab-7ea8-4c6e-9ac9-50a1a2799a29 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.102531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.103121 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.103393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.103653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.104281 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.106259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.136160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1316/2669] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:57.141107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.151806 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd9a12-bd65-4bc0-bc92-181c38f0f504 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.208853 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.209490 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.209767 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.210035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.210645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.212576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.234584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1354/2670] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:57:57.238808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.250094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f45b9-f8d3-4c4f-a98a-6475d8f0b71e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.256042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.256593 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.256863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.257166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.257797 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.259440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.270629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1317/2671] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:57.274859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.285446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60bd27e-de67-449e-98af-f0ed0013dbc3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.291974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.292608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.292901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.293183 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.293866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.295516 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81ae92-8b73-47dc-b94c-90fab7febaac admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.309570 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1355/2672] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:57.313853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.324396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61c1d2-5d4e-4215-ac5b-ebacffb5dbca admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.329118 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.329780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.330101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.330357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] PATH_INFO: `/v3/projects/2b3cfd515e0944e4917bfb1a84a9bd69/users/15455b31f49c4874b51e8740bb29b99d/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.348536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b3cfd515e0944e4917bfb1a84a9bd69, user_id=15455b31f49c4874b51e8740bb29b99d, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.351116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.365566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1318/2673] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:57 2022] PUT /identity/v3/projects/2b3cfd515e0944e4917bfb1a84a9bd69/users/15455b31f49c4874b51e8740bb29b99d/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:57.377441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.377807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.378146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.410228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] MFA Rules not processed for user `15455b31f49c4874b51e8740bb29b99d`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:57.411496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:57.439756 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:57.483757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1356/2674] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:57.488638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.552132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d742e-264a-47aa-8dc8-247dbf9f5471 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.557465 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.558190 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.558469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.558740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.559346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.561316 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.592406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1319/2675] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:57.594222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.605666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82604ca8-784d-4839-ab40-dbfc9f969887 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.665497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.666109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.666401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.666654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.667448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.669397 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.692415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1357/2676] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:57:57.696835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.707885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c02a2-cf1c-49b7-802a-b226060da9f3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.713339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.713941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.714214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.714476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.715143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.716742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.728737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1320/2677] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:57.733367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.743698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49380912-d163-4c87-beb8-72286d42d2fe admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.749166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.749830 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.750102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.750366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] PATH_INFO: `/v3/projects/0917c4540cf64e27995c706d301e8ce7/users/46619342125e406cadfdfcac1a425575/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.766249 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0917c4540cf64e27995c706d301e8ce7, user_id=46619342125e406cadfdfcac1a425575, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.768917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.783964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1358/2678] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:57 2022] PUT /identity/v3/projects/0917c4540cf64e27995c706d301e8ce7/users/46619342125e406cadfdfcac1a425575/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:57.788113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.798920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652e99d-1375-499f-a90b-2dd925c9045d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.855261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.855961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.856154 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.856428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.857110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.858730 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5957c62f-8b45-4109-bc6c-76af4a525818 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.868756 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1321/2679] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:57:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:57:57.873378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:57.890068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc055e-57d8-4b97-a19c-60f4a8ba18aa admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:57.895012 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:57.895680 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.895988 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.896258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] PATH_INFO: `/v3/projects/0917c4540cf64e27995c706d301e8ce7/users/46619342125e406cadfdfcac1a425575/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.898237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0917c4540cf64e27995c706d301e8ce7, user_id=46619342125e406cadfdfcac1a425575, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:57.900878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:57.911868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1359/2680] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:57:57 2022] PUT /identity/v3/projects/0917c4540cf64e27995c706d301e8ce7/users/46619342125e406cadfdfcac1a425575/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:57.922978 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:57.923274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:57.923554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:57.931932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:57.953034 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] MFA Rules not processed for user `46619342125e406cadfdfcac1a425575`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:57.954257 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:58.023312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1322/2681] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:58.036051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:58.037894 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:58.097371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cc6a0-3706-4d47-b3b7-b8a894e4d6a1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:58.102070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:58.102686 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:58.102963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:58.103221 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:58.113954 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:58.115325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:58.125496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1360/2682] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:58.461192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:58.471020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c5c01-50ad-4f83-b1a4-9e8244c91a2f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:58.476429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46619342125e406cadfdfcac1a425575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0917c4540cf64e27995c706d301e8ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:58.477122 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:58.477438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:58.477714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:58.478405 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:58.478716 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:58.480431 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:58.493684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1323/2683] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:58.497964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:58.507520 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e6eaf8-c50d-4070-9959-2c15ad132fab service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:58.512226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46619342125e406cadfdfcac1a425575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0917c4540cf64e27995c706d301e8ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:58.512968 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:58.513161 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:58.513437 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:58.514087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:58.516006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:58.539459 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:58.549038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1361/2684] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:57:58 2022] POST /identity/v3/projects => generated 354 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:57:58.553064 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:58.563591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92618f21-3ac4-415d-8067-dd32806850e9 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:58.627813 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46619342125e406cadfdfcac1a425575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0917c4540cf64e27995c706d301e8ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:58.628424 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:58.628740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:58.629059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:58.629733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:57:58.630036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:58.631686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:58.644608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1324/2685] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:57:58.691562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:58.703154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5680168-b383-4580-b67b-9f56dc84becd tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:58.707789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46619342125e406cadfdfcac1a425575', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0917c4540cf64e27995c706d301e8ce7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:58.708493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:58.708771 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:58.709065 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] PATH_INFO: `/v3/projects/29a865a7afdb44a99e65f3fd01bc07ca` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:58.710186 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorizing `identity:delete_project(project_id=29a865a7afdb44a99e65f3fd01bc07ca)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:58.712108 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:58.763154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a865a7afdb44a99e65f3fd01bc07ca', 'request_id': 'req-3c026de5-9340-47d4-80af-275915ee72c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:58.770617 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a865a7afdb44a99e65f3fd01bc07ca', 'request_id': 'req-3c026de5-9340-47d4-80af-275915ee72c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:58.780339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a865a7afdb44a99e65f3fd01bc07ca', 'request_id': 'req-3c026de5-9340-47d4-80af-275915ee72c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:58.791546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] The token cache is being invalidate because project 29a865a7afdb44a99e65f3fd01bc07ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:58.791837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:58.792500 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-3c026de5-9340-47d4-80af-275915ee72c7 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:58.794064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1362/2686] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:58 2022] DELETE /identity/v3/projects/29a865a7afdb44a99e65f3fd01bc07ca => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:59.014458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:59.016720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:59.088346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941902a9-a5f2-4bd8-8cba-3e4346740ba5 tempest-VolumeQuotaClassesTest-1592284841 tempest-VolumeQuotaClassesTest-1592284841-project-admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:59.101699 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:59.148560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:59.149218 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:59.149518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:59.149790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] PATH_INFO: `/v3/users/15455b31f49c4874b51e8740bb29b99d` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:59.150936 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15455b31f49c4874b51e8740bb29b99d)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:59.152988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:59.202777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15455b31f49c4874b51e8740bb29b99d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.211206 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15455b31f49c4874b51e8740bb29b99d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.211925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15455b31f49c4874b51e8740bb29b99d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.212580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15455b31f49c4874b51e8740bb29b99d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.225193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15455b31f49c4874b51e8740bb29b99d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.238290 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1325/2687] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:59 2022] DELETE /identity/v3/users/15455b31f49c4874b51e8740bb29b99d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:59.243174 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:59.243668 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:59.243791 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:59.265030 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:57:59.266656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:59.330079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1363/2688] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:57:59.340771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:59.342412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:59.402484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979990a-9871-403d-979e-014865a11f7c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:59.406296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:59.406899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:59.407173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:59.407439 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:59.417236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:59.418669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:59.428308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1326/2689] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:59.713524 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:59.715114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:59.728614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c78768-a78c-4608-b17c-cae98060e668 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:59.733591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:59.734380 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:59.734750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:59.735140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] PATH_INFO: `/v3/projects/2b3cfd515e0944e4917bfb1a84a9bd69` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:59.736508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b3cfd515e0944e4917bfb1a84a9bd69)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:59.738558 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:59.792831 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b3cfd515e0944e4917bfb1a84a9bd69', 'request_id': 'req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.800040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b3cfd515e0944e4917bfb1a84a9bd69', 'request_id': 'req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.808682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b3cfd515e0944e4917bfb1a84a9bd69', 'request_id': 'req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.819627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] The token cache is being invalidate because project 2b3cfd515e0944e4917bfb1a84a9bd69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:57:59.819913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.820542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.822409 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1364/2690] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:57:59 2022] DELETE /identity/v3/projects/2b3cfd515e0944e4917bfb1a84a9bd69 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:57:59.826651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:59.836296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303eed41-9ea0-4ff2-9355-d7f35dd7b522 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:59.874850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:59.895325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:59.895950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:59.896221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:59.896486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] PATH_INFO: `/v3/users/46619342125e406cadfdfcac1a425575` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:59.897623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46619342125e406cadfdfcac1a425575)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:59.899543 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:59.948870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46619342125e406cadfdfcac1a425575'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.957571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46619342125e406cadfdfcac1a425575'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.958256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46619342125e406cadfdfcac1a425575'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.958811 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46619342125e406cadfdfcac1a425575'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.970841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b91f9eda-5f67-4ac6-97eb-6ee90b116310 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46619342125e406cadfdfcac1a425575'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:57:59.983318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1327/2691] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:57:59 2022] DELETE /identity/v3/users/46619342125e406cadfdfcac1a425575 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:00.265763 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.267323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:00.277253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:00.281482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3e334-0f33-4b7f-8e54-a658669c4e1f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.344055 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.344635 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.344941 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.345264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] PATH_INFO: `/v3/projects/0917c4540cf64e27995c706d301e8ce7` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.346370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0917c4540cf64e27995c706d301e8ce7)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.348246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.394035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0917c4540cf64e27995c706d301e8ce7', 'request_id': 'req-09cd1647-2405-4748-9460-0708e9c4af67'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:00.400236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0917c4540cf64e27995c706d301e8ce7', 'request_id': 'req-09cd1647-2405-4748-9460-0708e9c4af67'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:00.409576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0917c4540cf64e27995c706d301e8ce7', 'request_id': 'req-09cd1647-2405-4748-9460-0708e9c4af67'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:00.420724 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] The token cache is being invalidate because project 0917c4540cf64e27995c706d301e8ce7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:00.421039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:00.421915 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-09cd1647-2405-4748-9460-0708e9c4af67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:00.423867 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1365/2692] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:00 2022] DELETE /identity/v3/projects/0917c4540cf64e27995c706d301e8ce7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:00.429099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.429387 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.429660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.450274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:00.451497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:00.513272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1328/2693] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:00.521908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.522187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.522431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.544816 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:00.546106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:00.554866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1366/2694] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:00.559587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.570224 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568ea4a3-05cf-4bec-b5aa-21b017c7636e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.575800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.576406 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.576687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.576997 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.577639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:00.577955 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.579564 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.592069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1329/2695] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:00.596704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.606580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d07c4-bf87-4e84-8ab4-6f88f227796e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.612532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.613209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.613475 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.613733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.614358 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.616260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.647918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1367/2696] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:00 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:00.652953 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.664041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8524b37-4fbd-4b9e-89b2-f19d8711326e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.714539 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:00.729727 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.730306 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.730574 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.730841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.731477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.733499 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.758772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1330/2697] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:00 2022] POST /identity/v3/users => generated 289 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:00.763654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.774944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12640068-5e36-45f8-b8f8-2cdea66f188b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.780265 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.780904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.781195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.781465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.782115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.783736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.795601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1368/2698] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:00.800040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.810812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7eca27-a247-4d17-868c-b58ac9b6f7b6 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.816286 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.816927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.817232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.817502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.818151 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.819793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a9260-7df9-4d80-b309-dde24d9d3ee1 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.833438 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1331/2699] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:00.837765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:00.851017 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46531a31-e8d6-41db-bfb9-0d14e4d55fd7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:00.856027 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:00.856716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.857037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.857312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] PATH_INFO: `/v3/projects/927b21ca9cd14cab8a64c8a37cc78379/users/cfc1103dc6d447979a345f40bfed5966/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.874515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=927b21ca9cd14cab8a64c8a37cc78379, user_id=cfc1103dc6d447979a345f40bfed5966, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:00.877376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:00.892718 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1369/2700] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:00 2022] PUT /identity/v3/projects/927b21ca9cd14cab8a64c8a37cc78379/users/cfc1103dc6d447979a345f40bfed5966/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:00.903429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:00.903801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:00.904075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:00.935113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] MFA Rules not processed for user `cfc1103dc6d447979a345f40bfed5966`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:00.936425 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:01.012707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1332/2701] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:01.017402 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.057140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:01.089212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d130789-89e5-4fce-b29d-28cf62b60ee4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.094009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.094627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.094913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.095176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.095810 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.097757 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.128210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1370/2702] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:01.132762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.144239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d078313-d02c-4fb9-8efd-a81a03d9f48a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.203559 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.204179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.204455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.204726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.205396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.207322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.231171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1333/2703] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:01.238018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.249248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf6434-4bbe-4df7-bf21-4b968c6acd79 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.254426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.255293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.255717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.255985 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.256681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.258673 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.270921 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1371/2704] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:01.274912 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.285768 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace764d-c6d8-4a5b-8415-ca2952e8062d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.290986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.291617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.291890 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.292211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.292859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.294527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e503ac-3fdc-4923-89b1-74bb76b80d75 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.305123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1334/2705] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:01.309667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.322312 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ed8b4-c0b0-49f5-989a-5147996a750f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.329970 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.330665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.330939 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.331212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] PATH_INFO: `/v3/projects/ad461758cef94c91b1debbd8832e302b/users/2c7391ac21494b1b8f078f607be9063e/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.348381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad461758cef94c91b1debbd8832e302b, user_id=2c7391ac21494b1b8f078f607be9063e, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.351079 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.365418 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1372/2706] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:01 2022] PUT /identity/v3/projects/ad461758cef94c91b1debbd8832e302b/users/2c7391ac21494b1b8f078f607be9063e/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:01.376644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.376916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.377221 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.408943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] MFA Rules not processed for user `2c7391ac21494b1b8f078f607be9063e`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:01.410164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:01.434322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:01.483198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1335/2707] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:01.487396 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.551068 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d78771-611b-4981-a091-1ce21fc04ff1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.557554 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.558175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.558447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.558712 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.559346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.561352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.594271 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1373/2708] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:01.598612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.609881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d4ce1-032c-48f3-aa30-119398e4137b None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.669992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.670505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.670777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.671039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.671669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.673605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.700921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1336/2709] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:01.705126 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.715865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45596407-ffb5-4f57-b939-a28787cbd245 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.720621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.721257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.721537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.721808 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.722448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.724717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.737614 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1374/2710] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:01.742062 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.753973 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb847fa8-0d5d-4bc2-9456-5e1bbc9178da admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.758794 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.759470 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.759745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.760026 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] PATH_INFO: `/v3/projects/fd043f531975411e96f1d463d26c2246/users/2b4eb7da60a446d39a4010e87eb360c9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.777336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd043f531975411e96f1d463d26c2246, user_id=2b4eb7da60a446d39a4010e87eb360c9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.779986 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.795204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1337/2711] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:01 2022] PUT /identity/v3/projects/fd043f531975411e96f1d463d26c2246/users/2b4eb7da60a446d39a4010e87eb360c9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:01.799245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.809609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a420ee-c35d-4fb6-9a60-fb18803dfa0b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.865960 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:01.875212 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.875880 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.876425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.877083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.877734 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.879355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e1977c-f5af-49a2-a82b-3d02bd5ea117 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.890610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1375/2712] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:01.894919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:01.905821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0205ac7-8cc1-4c21-8788-85463df75496 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:01.910586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:01.911253 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.911530 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.911793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] PATH_INFO: `/v3/projects/fd043f531975411e96f1d463d26c2246/users/2b4eb7da60a446d39a4010e87eb360c9/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.913852 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd043f531975411e96f1d463d26c2246, user_id=2b4eb7da60a446d39a4010e87eb360c9, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:01.916841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:01.927830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1338/2713] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:01 2022] PUT /identity/v3/projects/fd043f531975411e96f1d463d26c2246/users/2b4eb7da60a446d39a4010e87eb360c9/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:01.938238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:01.938542 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:01.938809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:01.972087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] MFA Rules not processed for user `2b4eb7da60a446d39a4010e87eb360c9`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:01.973386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:02.064222 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1376/2714] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:02.076136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:02.078020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:02.147535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae1fa7-cb12-4093-9062-46837ae9e9a3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:02.151302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:02.152030 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:02.152335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:02.152602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:02.162220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:02.163651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:02.174536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1339/2715] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:02.217710 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:02.219185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:02.231357 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d2217c-5b01-4236-8fde-0221f82058f0 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:02.235426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:02.236032 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:02.236320 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:02.236568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:02.295631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:02.297103 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178e99b9-a153-4e52-a237-d95779d7c7dd service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:02.306956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1377/2716] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:05.325155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:05.352283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:05.407440 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200933f-611a-4313-b0a7-1ff4275972df service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:05.412283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:05.413039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:05.413366 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:05.413746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] PATH_INFO: `/v3/users/cfc1103dc6d447979a345f40bfed5966` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:05.415331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfc1103dc6d447979a345f40bfed5966)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:05.417637 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:05.477454 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfc1103dc6d447979a345f40bfed5966'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:05.487871 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfc1103dc6d447979a345f40bfed5966'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:05.488740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfc1103dc6d447979a345f40bfed5966'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:05.489648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfc1103dc6d447979a345f40bfed5966'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:05.504227 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfc1103dc6d447979a345f40bfed5966'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:05.517920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1340/2717] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:05 2022] DELETE /identity/v3/users/cfc1103dc6d447979a345f40bfed5966 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:05.522714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:05.523307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:05.523307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:05.546086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:05.548166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:05.638907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1378/2718] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:05.650152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:05.651919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:05.742720 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d93b7c0-123e-47f4-aa4e-fc94978e45fb admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:05.747272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:05.748044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:05.748408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:05.748772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:05.759881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:05.762080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:05.775691 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1341/2719] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:06.116433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:06.127546 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd2ea1-3f1d-483a-81d2-c8d9b9244b94 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:06.131912 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:06.132869 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:06.133311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:06.133804 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:06.136422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:06.151311 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=100 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:06.215625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:06.217116 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:06.228681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1379/2720] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:06.246744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:06.249596 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:06.283870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f124fb5a-ad85-4632-9c35-00c8c76b5b71 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:06.293180 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:06.294260 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:06.294877 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:06.295416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] PATH_INFO: `/v3/projects/927b21ca9cd14cab8a64c8a37cc78379` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:06.297577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927b21ca9cd14cab8a64c8a37cc78379)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:06.301580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:06.374132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:06.389614 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927b21ca9cd14cab8a64c8a37cc78379', 'request_id': 'req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.400331 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927b21ca9cd14cab8a64c8a37cc78379', 'request_id': 'req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.412696 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927b21ca9cd14cab8a64c8a37cc78379', 'request_id': 'req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.427903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] The token cache is being invalidate because project 927b21ca9cd14cab8a64c8a37cc78379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:06.428251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.429078 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.431186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1342/2721] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:06 2022] DELETE /identity/v3/projects/927b21ca9cd14cab8a64c8a37cc78379 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:06.436633 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:06.454654 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3d217-82b4-402e-937f-b0bf1e294bdf service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:06.535325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:06.536504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:06.536994 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:06.537415 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] PATH_INFO: `/v3/users/2c7391ac21494b1b8f078f607be9063e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:06.538904 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7391ac21494b1b8f078f607be9063e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:06.541716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:06.610950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7391ac21494b1b8f078f607be9063e'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.622760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7391ac21494b1b8f078f607be9063e'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.623783 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7391ac21494b1b8f078f607be9063e'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.624584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7391ac21494b1b8f078f607be9063e'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.634095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7391ac21494b1b8f078f607be9063e'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:06.656541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1380/2722] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:06 2022] DELETE /identity/v3/users/2c7391ac21494b1b8f078f607be9063e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:07.206226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.208476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.226004 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6382d37b-4d0c-44cf-a7b0-0607fbd3c2d7 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.338038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.339289 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.339780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.340254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] PATH_INFO: `/v3/projects/ad461758cef94c91b1debbd8832e302b` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.342785 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad461758cef94c91b1debbd8832e302b)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.346726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.423059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad461758cef94c91b1debbd8832e302b', 'request_id': 'req-852f0b56-e127-43fe-9ca6-9016372fde1a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.435565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad461758cef94c91b1debbd8832e302b', 'request_id': 'req-852f0b56-e127-43fe-9ca6-9016372fde1a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.449108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad461758cef94c91b1debbd8832e302b', 'request_id': 'req-852f0b56-e127-43fe-9ca6-9016372fde1a'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.473035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] The token cache is being invalidate because project ad461758cef94c91b1debbd8832e302b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:07.473565 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.474793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.477494 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1343/2723] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:07 2022] DELETE /identity/v3/projects/ad461758cef94c91b1debbd8832e302b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:07.481914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:07.483616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:07.500219 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d21e2-dfad-4cc8-9ad2-3e0c035b078d admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:07.543039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:07.585118 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:07.585871 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:07.586208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:07.586552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] PATH_INFO: `/v3/users/2b4eb7da60a446d39a4010e87eb360c9` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:07.588072 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4eb7da60a446d39a4010e87eb360c9)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:07.590557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:07.662095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4eb7da60a446d39a4010e87eb360c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.676793 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4eb7da60a446d39a4010e87eb360c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.677819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4eb7da60a446d39a4010e87eb360c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.678795 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4eb7da60a446d39a4010e87eb360c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.692154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8ab302f6-995b-4f13-bab2-990c1639001c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4eb7da60a446d39a4010e87eb360c9'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:07.716938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1381/2724] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:07 2022] DELETE /identity/v3/users/2b4eb7da60a446d39a4010e87eb360c9 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:08.094909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.096476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.116579 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f0b56-e127-43fe-9ca6-9016372fde1a admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.180758 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.194900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.195711 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.195994 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.196270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] PATH_INFO: `/v3/projects/fd043f531975411e96f1d463d26c2246` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.197507 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd043f531975411e96f1d463d26c2246)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.199571 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.264020 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd043f531975411e96f1d463d26c2246', 'request_id': 'req-5225491e-3675-48aa-999b-2e3d6e5524ca'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:08.274468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd043f531975411e96f1d463d26c2246', 'request_id': 'req-5225491e-3675-48aa-999b-2e3d6e5524ca'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:08.287509 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd043f531975411e96f1d463d26c2246', 'request_id': 'req-5225491e-3675-48aa-999b-2e3d6e5524ca'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:08.301437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] The token cache is being invalidate because project fd043f531975411e96f1d463d26c2246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:08.301732 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:08.302396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-5225491e-3675-48aa-999b-2e3d6e5524ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:08.303951 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1344/2725] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:08 2022] DELETE /identity/v3/projects/fd043f531975411e96f1d463d26c2246 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:08.312807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.313321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.313828 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.343259 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.344646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.427693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1382/2726] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.429578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.430194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.430194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.454050 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.455377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:08.467603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1345/2727] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:08.469253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.480747 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f6096-491f-48c4-a41d-290601a785d7 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.486026 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.486656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.486928 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.487194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.487851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:08.488152 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.489788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.505625 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1383/2728] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:08.510135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.534216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbef8788-f175-46d4-9f25-4b6ca0212000 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.539880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.540919 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.541214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.541490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.542412 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.544796 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.583725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1346/2729] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:08.588158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.599568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a17e32-80ac-4f9b-850a-f41802f95a71 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.669289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:08.718703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.719366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.719742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.719924 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.721134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.722543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.748115 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1384/2730] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:08.753128 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.765667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826db63-66ad-4ed2-8457-6f9e209a1023 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.771103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.771748 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.772023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.772318 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.773822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.774619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.785982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1347/2731] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:08.790695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.801555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019643a-e290-49b5-bdd0-cf810c3e54be admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.806315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.806949 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.807293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.807766 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.808499 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.810251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a8da57-febd-42b0-ab0d-c39d8730c877 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.825169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1385/2732] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:08.826897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:08.838772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fd3c2-fba4-48c1-955e-62253f7c726f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:08.846659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:08.847484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.847819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.848155 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] PATH_INFO: `/v3/projects/e496fe8b29dc40cdbf59dd5005242a19/users/fc36a9465a1044c5b17da800493a83c7/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.866082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e496fe8b29dc40cdbf59dd5005242a19, user_id=fc36a9465a1044c5b17da800493a83c7, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:08.868788 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:08.892228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1348/2733] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:08 2022] PUT /identity/v3/projects/e496fe8b29dc40cdbf59dd5005242a19/users/fc36a9465a1044c5b17da800493a83c7/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:08.908859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:08.909374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:08.909861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:08.950651 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] MFA Rules not processed for user `fc36a9465a1044c5b17da800493a83c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:08.952173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.056256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1386/2734] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:09.061116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.144558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90a61d-3e24-49b3-b1e9-ea82345154b8 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.150102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.151089 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.151375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.151684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.152341 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.154442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.192267 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1349/2735] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.197550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.210348 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc2825-f478-4a9d-af59-012e6b3e1c85 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.281583 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.282220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.282498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.282765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.283428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.285403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.313761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1387/2736] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:09.318351 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.328111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.333296 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8214a-103d-4cff-b8fb-d1de144a87fd admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.343018 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.344005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.344434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.345080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.346477 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.349148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.362902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1350/2737] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:09.367282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.377750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6810f-2a24-4467-ad43-79af84255fe8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.382990 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.383715 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.383996 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.384268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] PATH_INFO: `/v3/projects/f86e7374eb8d4d11b4b3e4a509201705/users/a581e4363cf54b7b8e7bbfd89775dc8f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.401340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86e7374eb8d4d11b4b3e4a509201705, user_id=a581e4363cf54b7b8e7bbfd89775dc8f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.404208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.427208 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1388/2738] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/f86e7374eb8d4d11b4b3e4a509201705/users/a581e4363cf54b7b8e7bbfd89775dc8f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.433047 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.447153 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85308d-cdb1-412f-a113-e9fef6217bb0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.531792 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.532827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.533147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.533533 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.534265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.536211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685faa4-c4b9-4a2a-8a9f-c7d5354c9eda admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.549688 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1351/2739] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:09.554556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.567042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0fd30-9c23-489e-bb25-0255fbce2fda admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.571893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.572579 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.572856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.573159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] PATH_INFO: `/v3/projects/f86e7374eb8d4d11b4b3e4a509201705/users/a581e4363cf54b7b8e7bbfd89775dc8f/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.575403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f86e7374eb8d4d11b4b3e4a509201705, user_id=a581e4363cf54b7b8e7bbfd89775dc8f, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.578066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.581628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:09.593703 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1389/2740] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:09 2022] PUT /identity/v3/projects/f86e7374eb8d4d11b4b3e4a509201705/users/a581e4363cf54b7b8e7bbfd89775dc8f/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:09.612535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.613107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.613575 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.650673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] MFA Rules not processed for user `a581e4363cf54b7b8e7bbfd89775dc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:09.651987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.750116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1352/2741] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:09.762547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:09.764519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:09.825716 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f952672-403f-4228-8930-7db9c5ad64fa admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:09.829552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:09.830192 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:09.830449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:09.830713 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:09.889799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:09.891287 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:09.900837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1390/2742] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:14.473302 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:14.475531 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:14.528363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0323f669-0785-4ab6-8826-a1fbb33b826f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:14.532679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:14.533361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:14.533641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:14.533909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:14.572113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:14.573710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a744c0a-a8b1-41d4-9097-fb0557364542 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:14.585771 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1353/2743] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:19.733083 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:19.893774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7871427b-9a00-4ff5-aa92-8ec58a3db46c service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:19.898897 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:19.899565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:19.899842 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:19.900112 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] PATH_INFO: `/v3/users/fc36a9465a1044c5b17da800493a83c7` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:19.901410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc36a9465a1044c5b17da800493a83c7)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:19.903506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:19.923846 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:19.975575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc36a9465a1044c5b17da800493a83c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.990024 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc36a9465a1044c5b17da800493a83c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.991131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc36a9465a1044c5b17da800493a83c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:19.991907 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc36a9465a1044c5b17da800493a83c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.002580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc36a9465a1044c5b17da800493a83c7'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.047349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1391/2744] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:19 2022] DELETE /identity/v3/users/fc36a9465a1044c5b17da800493a83c7 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.054310 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.055109 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.055284 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.104926 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:20.147870 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:20.149124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.243475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1354/2745] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:20.256934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.258479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.341567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a50af-23d2-4baa-aebe-ebed2e61cdca admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.346600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.347268 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.347547 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.347817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.358481 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.359989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.371526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1392/2746] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:20.747835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.755737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:20.774043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01a564-1eba-4e78-8cc3-b263d0c13a8e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:20.794396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:20.795114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:20.795472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:20.795630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] PATH_INFO: `/v3/projects/e496fe8b29dc40cdbf59dd5005242a19` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:20.797148 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e496fe8b29dc40cdbf59dd5005242a19)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:20.799436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:20.872300 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e496fe8b29dc40cdbf59dd5005242a19', 'request_id': 'req-1c670d4c-4cc4-43c1-b810-35e478599f24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.885211 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e496fe8b29dc40cdbf59dd5005242a19', 'request_id': 'req-1c670d4c-4cc4-43c1-b810-35e478599f24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.905309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e496fe8b29dc40cdbf59dd5005242a19', 'request_id': 'req-1c670d4c-4cc4-43c1-b810-35e478599f24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.926098 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] The token cache is being invalidate because project e496fe8b29dc40cdbf59dd5005242a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:20.926558 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.927562 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:20.929981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1355/2747] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:20 2022] DELETE /identity/v3/projects/e496fe8b29dc40cdbf59dd5005242a19 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:20.935574 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:20.946422 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56632837-e76f-474b-86d0-673c9d851abe service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.036704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.037732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.038139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.038584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] PATH_INFO: `/v3/users/a581e4363cf54b7b8e7bbfd89775dc8f` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.041060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a581e4363cf54b7b8e7bbfd89775dc8f)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.045036 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.109963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:21.121962 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a581e4363cf54b7b8e7bbfd89775dc8f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.136016 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a581e4363cf54b7b8e7bbfd89775dc8f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.137051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a581e4363cf54b7b8e7bbfd89775dc8f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.137849 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a581e4363cf54b7b8e7bbfd89775dc8f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.150476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6a55dffa-05ba-4ad2-b096-a75a560dda24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a581e4363cf54b7b8e7bbfd89775dc8f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.186207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1393/2748] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:20 2022] DELETE /identity/v3/users/a581e4363cf54b7b8e7bbfd89775dc8f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:21.604008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:21.606108 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.627434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c670d4c-4cc4-43c1-b810-35e478599f24 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:21.713075 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:21.713907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.714232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.714542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] PATH_INFO: `/v3/projects/f86e7374eb8d4d11b4b3e4a509201705` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.715988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f86e7374eb8d4d11b4b3e4a509201705)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:21.718045 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:21.750002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:21.785433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f86e7374eb8d4d11b4b3e4a509201705', 'request_id': 'req-d7289f08-e960-497b-9c95-6b102a669f20'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.794256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f86e7374eb8d4d11b4b3e4a509201705', 'request_id': 'req-d7289f08-e960-497b-9c95-6b102a669f20'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.805022 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f86e7374eb8d4d11b4b3e4a509201705', 'request_id': 'req-d7289f08-e960-497b-9c95-6b102a669f20'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.825876 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] The token cache is being invalidate because project f86e7374eb8d4d11b4b3e4a509201705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:21.826344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.827468 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d7289f08-e960-497b-9c95-6b102a669f20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:21.830058 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1356/2749] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:21 2022] DELETE /identity/v3/projects/f86e7374eb8d4d11b4b3e4a509201705 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:21.836841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:21.837288 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:21.837665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:21.906709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:21.908092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:21.996553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1394/2750] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:22.009244 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.009556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.009862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.033008 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:22.034469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.045683 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1357/2751] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:22.050693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.061896 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90764b-cdf2-4e0e-8f48-bd28f53fd5de None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.067275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.068144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.068675 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.069059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.070844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:22.071260 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.073127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.088253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1395/2752] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:58:22.094234 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.109017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce865a4-a267-49c1-907b-45ade96ca4e7 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.114511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.115299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.115580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.115845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.116490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.118466 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.153629 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1358/2753] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:22.163171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.177207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166557e-53f7-467f-8ea4-a10a62a56240 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.249423 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.250058 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.250339 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.250607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.251282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.253341 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.279893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1396/2754] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:22.284479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.297900 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39e68dc-3af5-496f-a5d5-b9210bfd3e3e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.303461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.304101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.304381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.304652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.305361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.307036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.320563 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1359/2755] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:22.325992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.341578 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777afde8-df15-4e0e-9015-0532ae24b387 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.349748 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.350773 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.351628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.351798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.352844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.356294 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f50732-f5bc-45c1-b7c5-c8ce38d38493 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.361427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:22.379977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1397/2756] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:22.381560 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.395171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bf277-fecd-4d76-ab16-549e11cb70fe admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.402925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.403723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.404100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.404523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] PATH_INFO: `/v3/projects/9d3eb509bf1b46d99dddbc030d51d18e/users/9b1fc3a959544ee5b5b401b1f457caa2/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.425988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3eb509bf1b46d99dddbc030d51d18e, user_id=9b1fc3a959544ee5b5b401b1f457caa2, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.429467 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.446763 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1360/2757] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:22 2022] PUT /identity/v3/projects/9d3eb509bf1b46d99dddbc030d51d18e/users/9b1fc3a959544ee5b5b401b1f457caa2/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:22.546044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.546360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.546674 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.589809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] MFA Rules not processed for user `9b1fc3a959544ee5b5b401b1f457caa2`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:22.591237 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:22.698207 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1398/2758] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:22.705349 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.797080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f57dbc-d0f7-45af-b088-ea2375457b3f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.810684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.811760 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.812236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.812747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.814479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.818198 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.853903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1361/2759] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:22.862551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.878113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206feb3a-1fe6-4e78-8979-7d3ebaaac5fb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:22.949856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:22.950535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:22.950809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:22.951078 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:22.951725 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:22.954127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:22.980011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1399/2760] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:22 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:22.986279 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:22.998822 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47680eb-1411-4c65-905f-464a86d54ab7 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.004982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.005937 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.006377 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.006854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.008144 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.011137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.033177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1362/2761] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.039662 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.054158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd6be4-5600-415a-865f-afecfa61903b admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.061637 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.062403 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.062741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.063070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.063801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.065641 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557eee5-651d-445b-a02d-791cd588d936 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.095304 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1400/2762] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.101119 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.112873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d98cb5-d7e3-45d7-bade-47a6eaa5a1b0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.117950 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.118643 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.118932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.119188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] PATH_INFO: `/v3/projects/e5fdc09588a44690b5dca1cddf7fc824/users/915fedc6fff14d7682b883441c7870f8/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.137343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5fdc09588a44690b5dca1cddf7fc824, user_id=915fedc6fff14d7682b883441c7870f8, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.140087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.144145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:23.157731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1363/2763] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:23 2022] PUT /identity/v3/projects/e5fdc09588a44690b5dca1cddf7fc824/users/915fedc6fff14d7682b883441c7870f8/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:23.169560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.169855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.170105 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.217447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] MFA Rules not processed for user `915fedc6fff14d7682b883441c7870f8`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:23.219555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:23.301131 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:23.317743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1401/2764] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:23.324055 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.417346 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4c4b-a095-42a3-b34b-381133f5a465 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.424204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.425249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.425725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.426187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] PATH_INFO: `/v3/projects` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.427390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.431188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.473472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1364/2765] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:23.478350 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.492815 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b08e59-2cbf-4c00-b6b8-bf7b41da1e99 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.611411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.612064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.612364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.612624 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] PATH_INFO: `/v3/users` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.613315 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.615327 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.641242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1402/2766] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/users => generated 298 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:23.644692 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.657861 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da85dca6-daae-4ff0-b4f3-8ba66f29b72d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.663537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.664343 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.664626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.664965 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.666012 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.667751 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.682107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1365/2767] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.686852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.699495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c988d7b-c87d-4031-8463-a940b0036813 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.705786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.706600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.706884 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.707158 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] PATH_INFO: `/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d/users/a638119517524bd48a24bc1d3e9142ab/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.732865 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0b1fa7f55c4cc5a9532b8e4f11439d, user_id=a638119517524bd48a24bc1d3e9142ab, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.735923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.757963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1403/2768] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:23 2022] PUT /identity/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d/users/a638119517524bd48a24bc1d3e9142ab/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:23.763259 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.779726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeb5f3b-5a88-4aa0-bd1a-85782e616a5e admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.863686 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.864475 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.864854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.865288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.866032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.868033 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460e5d8a-9e4e-4f41-84e0-18b7ecf39a04 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.886232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1366/2769] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:23.887588 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:23.903442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a21423-ce05-4c37-915e-dce8c47d5600 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:23.912400 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:23.913586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.914040 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.914497 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] PATH_INFO: `/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d/users/a638119517524bd48a24bc1d3e9142ab/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:23.918455 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0b1fa7f55c4cc5a9532b8e4f11439d, user_id=a638119517524bd48a24bc1d3e9142ab, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:23.924091 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:23.944517 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1404/2770] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:23 2022] PUT /identity/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d/users/a638119517524bd48a24bc1d3e9142ab/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:23.965095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:23.965568 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:23.966099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.018710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] MFA Rules not processed for user `a638119517524bd48a24bc1d3e9142ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:24.020776 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:24.108946 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:24.132097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1367/2771] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:24.149309 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:24.151458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:24.220373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95c1ce0-42a7-494d-9438-1f610f12c910 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:24.224768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:24.225463 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:24.225741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:24.226071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:24.326153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:24.328855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:24.343214 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1405/2772] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:26.084390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.097963 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad012a0a-c30b-4004-8f09-6d48206f8bd4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.103744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.104496 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.104921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.105367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.199612 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.201490 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.212345 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1368/2773] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:26.871387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:26.878425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:26.887855 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6fa754-b9f8-418b-832a-5a13818a0cd2 service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:26.895537 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:26.896560 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:26.897095 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:26.897492 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:26.914217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:26.917352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac89f2-7e3f-4b79-b9c1-6ba0f993a598 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:26.935728 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1406/2774] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:31.125192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:31.226099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0441c05-1434-4427-a895-c2a4809a39e0 service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:31.232093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:31.233437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:31.233922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:31.234408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] PATH_INFO: `/v3/users/9b1fc3a959544ee5b5b401b1f457caa2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:31.236436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b1fc3a959544ee5b5b401b1f457caa2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:31.239514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:31.307644 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b1fc3a959544ee5b5b401b1f457caa2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:31.317520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b1fc3a959544ee5b5b401b1f457caa2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:31.318217 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b1fc3a959544ee5b5b401b1f457caa2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:31.318848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b1fc3a959544ee5b5b401b1f457caa2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:31.334855 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b1fc3a959544ee5b5b401b1f457caa2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:31.356365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1369/2775] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:31 2022] DELETE /identity/v3/users/9b1fc3a959544ee5b5b401b1f457caa2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:31.363457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:31.364111 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:31.364410 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:31.396529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:31.397966 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:31.486296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1407/2776] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:31.500054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:31.502239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:31.530623 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:31.657039 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5a4d8-c1c3-4a03-a124-9e6e67dee60c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:31.661578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:31.662226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:31.662595 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:31.662859 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:31.675854 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:31.678703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcfc24-57b7-4d2d-89b6-e147ae64ddc4 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:31.697010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1370/2777] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:32.166916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.168877 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.187064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e8e9f4-36a0-48c6-b268-6e974d03ea4e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.195450 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:32.196543 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.197067 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.197544 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] PATH_INFO: `/v3/projects/9d3eb509bf1b46d99dddbc030d51d18e` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.199854 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3eb509bf1b46d99dddbc030d51d18e)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:32.203957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:32.298737 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3eb509bf1b46d99dddbc030d51d18e', 'request_id': 'req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.314502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3eb509bf1b46d99dddbc030d51d18e', 'request_id': 'req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.318035 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.318670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.318670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.329705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3eb509bf1b46d99dddbc030d51d18e', 'request_id': 'req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.342505 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] MFA Rules not processed for user `02bb08e3dee645e4abe2e6d5e106ea57`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:32.343772 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.346643 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] The token cache is being invalidate because project 9d3eb509bf1b46d99dddbc030d51d18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:32.347107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.348085 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.350913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1408/2778] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:32 2022] DELETE /identity/v3/projects/9d3eb509bf1b46d99dddbc030d51d18e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:32.355953 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.357902 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.422094 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1371/2779] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:32.452299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32808ef9-cebb-4873-b3c6-cf93fa2d92c8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:32.458200 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:32.459137 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:32.459472 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:32.459798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] PATH_INFO: `/v3/users/915fedc6fff14d7682b883441c7870f8` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:32.461319 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=915fedc6fff14d7682b883441c7870f8)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:32.464127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:32.484938 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:32.534318 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '915fedc6fff14d7682b883441c7870f8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.547910 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '915fedc6fff14d7682b883441c7870f8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.548851 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '915fedc6fff14d7682b883441c7870f8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.549762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '915fedc6fff14d7682b883441c7870f8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.560433 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '915fedc6fff14d7682b883441c7870f8'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:32.589660 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1409/2780] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:32 2022] DELETE /identity/v3/users/915fedc6fff14d7682b883441c7870f8 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:32.868774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:32.871249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:32.972152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8973a2fd-58e9-455b-b577-6d889b1e7b91 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:33.010087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:33.013364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:33.051749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:33.052450 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:33.052648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:33.052945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.055421 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:33.064619 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=100 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:33.067053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:33.069077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:33.079731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1372/2781] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:33.139227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b281d797-db47-4b2a-8e3b-7cb09f65a2c4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:33.144267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:33.144979 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:33.145289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:33.145562 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] PATH_INFO: `/v3/projects/e5fdc09588a44690b5dca1cddf7fc824` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.146812 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5fdc09588a44690b5dca1cddf7fc824)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:33.148778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:33.215860 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5fdc09588a44690b5dca1cddf7fc824', 'request_id': 'req-0ad346ca-2085-4a3d-be55-291c2eaa212f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.226687 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5fdc09588a44690b5dca1cddf7fc824', 'request_id': 'req-0ad346ca-2085-4a3d-be55-291c2eaa212f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.239022 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5fdc09588a44690b5dca1cddf7fc824', 'request_id': 'req-0ad346ca-2085-4a3d-be55-291c2eaa212f'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.254223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] The token cache is being invalidate because project e5fdc09588a44690b5dca1cddf7fc824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:33.254496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.255087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.256984 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1410/2782] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:33 2022] DELETE /identity/v3/projects/e5fdc09588a44690b5dca1cddf7fc824 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:33.262933 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:33.278041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db40e5a-e3cf-4f80-a82e-6f2999d8da71 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:33.359924 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:33.361101 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:33.361307 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:33.361795 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] PATH_INFO: `/v3/users/a638119517524bd48a24bc1d3e9142ab` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:33.363480 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=a638119517524bd48a24bc1d3e9142ab)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:33.366069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:33.439379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a638119517524bd48a24bc1d3e9142ab'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.453525 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a638119517524bd48a24bc1d3e9142ab'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.454680 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a638119517524bd48a24bc1d3e9142ab'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.455853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a638119517524bd48a24bc1d3e9142ab'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.481713 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a638119517524bd48a24bc1d3e9142ab'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:33.499649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1373/2783] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 19:58:33 2022] DELETE /identity/v3/users/a638119517524bd48a24bc1d3e9142ab => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:34.045205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.047707 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.065007 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad346ca-2085-4a3d-be55-291c2eaa212f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.099386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:34.161667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.162354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.162629 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.162901 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] PATH_INFO: `/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.164220 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0b1fa7f55c4cc5a9532b8e4f11439d)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.166623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.217134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.219668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.237840 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0b1fa7f55c4cc5a9532b8e4f11439d', 'request_id': 'req-0b542fde-12a8-4800-b7ad-323681ed9014'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:34.246538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0b1fa7f55c4cc5a9532b8e4f11439d', 'request_id': 'req-0b542fde-12a8-4800-b7ad-323681ed9014'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:34.257102 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0b1fa7f55c4cc5a9532b8e4f11439d', 'request_id': 'req-0b542fde-12a8-4800-b7ad-323681ed9014'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:34.271495 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] The token cache is being invalidate because project 6e0b1fa7f55c4cc5a9532b8e4f11439d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:58:34.271780 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:34.272478 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0b542fde-12a8-4800-b7ad-323681ed9014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:58:34.274176 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1411/2784] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 19:58:34 2022] DELETE /identity/v3/projects/6e0b1fa7f55c4cc5a9532b8e4f11439d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:34.280533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e16d-f51b-4d8e-91ab-3cdc757c0169 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.281045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e16d-f51b-4d8e-91ab-3cdc757c0169 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.281511 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107e16d-f51b-4d8e-91ab-3cdc757c0169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.309705 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-4107e16d-f51b-4d8e-91ab-3cdc757c0169 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:34.311454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-4107e16d-f51b-4d8e-91ab-3cdc757c0169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.378916 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:34.393888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a6569-f8ef-4a0e-99d7-27ef8244e2fa admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.412299 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1412/2785] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:34.422267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.422681 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.423039 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.447717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:34.449428 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.460196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1413/2786] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:34.467253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.482149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ba830-9075-4f03-825b-d8c1f8def642 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.488999 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.489987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.490389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.490592 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.490827 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.491356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.491794 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.492228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] PATH_INFO: `/v3/domains` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.493395 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93090) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:58:34.493840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:34.494048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.496745 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.515304 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.516663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1414/2787] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 19:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:58:34.517847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.523407 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.531237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1374/2788] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:34.538031 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa73e33-dba2-4c38-99ba-c5a516079876 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.546929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.547889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.548281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.548601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.549532 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.552931 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.589494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1415/2789] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:34.594103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.606120 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc84746-2508-4eab-b098-d3e68d12dccd service placement] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.718656 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.720235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.720708 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.721136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.721952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.724201 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.756833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1375/2790] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/users => generated 287 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:34.762206 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.780194 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eabf172-6c02-4bf8-b5f9-b2cee20646d8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.788523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.789597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.790050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.790556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.792013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.795153 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.805835 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:34.813381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1416/2791] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:34.818536 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.834684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17641cb-0e82-4bf3-a7e0-619b91646f40 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.845074 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.846176 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.846731 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.847276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.848540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.851482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbe05a-a6eb-4e06-9c4d-5b3e999621d9 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.877733 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1376/2792] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:34.882761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:34.896630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd963e7-56d7-41d6-ae62-b22f8272c6e5 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:34.905545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:34.906246 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.906528 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.906798 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] PATH_INFO: `/v3/projects/838511c93b924f118bcfc03ecfd870b2/users/9ca934350c71421cb644b3a380e96df2/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.928923 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=838511c93b924f118bcfc03ecfd870b2, user_id=9ca934350c71421cb644b3a380e96df2, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:34.934525 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:34.959424 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1417/2793] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:34 2022] PUT /identity/v3/projects/838511c93b924f118bcfc03ecfd870b2/users/9ca934350c71421cb644b3a380e96df2/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:58:34.967023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ff24-43f9-447e-a5ca-de7fc89c6f9f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:34.967309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ff24-43f9-447e-a5ca-de7fc89c6f9f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:34.967578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8ff24-43f9-447e-a5ca-de7fc89c6f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:34.991285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-56c8ff24-43f9-447e-a5ca-de7fc89c6f9f None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:34.992591 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-56c8ff24-43f9-447e-a5ca-de7fc89c6f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:35.101358 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1377/2794] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:35.115195 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:35.117159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:35.191468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80624f17-0aee-4615-9dbb-d397842bb0f7 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:35.195740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:35.196378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:35.196655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:35.196955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:35.207676 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:35.209374 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:35.221154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1418/2795] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:58:42.846215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:42.846805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:42.846805 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:42.864195 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:42.890324 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] MFA Rules not processed for user `9ca934350c71421cb644b3a380e96df2`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:42.891588 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:42.979099 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1378/2796] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:58:42.986151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:42.997209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b69d2-6466-4916-864d-a4def729dbcc service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.003382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.004066 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.004491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.004832 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.005822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.008829 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.062122 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1419/2797] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 19:58:42 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:58:43.063443 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.079633 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d95a8-ac54-4f70-8acf-c13c4abf23a4 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.174653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.175305 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.175583 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.175848 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.176514 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.178725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.207441 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1379/2798] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 19:58:43 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:58:43.212561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.225044 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67683a15-8ca4-4c5b-bdea-f2e47ded366f admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.230627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.231325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.231609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.231882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.232581 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.234342 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.247975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1420/2799] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:58:43.253373 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.265551 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598b68f-f5e7-42b7-a206-628803064a00 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.270921 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.271625 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.271902 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.272171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] PATH_INFO: `/v3/projects/2cb875ffae9841a78e3ff7212dd734f4/users/a623b62f1bd8448f95f7ac8db89d918e/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.289542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb875ffae9841a78e3ff7212dd734f4, user_id=a623b62f1bd8448f95f7ac8db89d918e, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.292517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.310761 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1380/2800] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:43 2022] PUT /identity/v3/projects/2cb875ffae9841a78e3ff7212dd734f4/users/a623b62f1bd8448f95f7ac8db89d918e/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:43.318353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.341362 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca440a28-b171-4732-a35c-ba35bf492472 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.434587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.435649 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.436117 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.436590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.437837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.441082 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9824307-01af-4bc5-bdec-6a9ddac6d5a2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.454474 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1421/2801] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 19:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:58:43.459209 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:43.472002 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de8ded-211d-4ab4-ba3b-58530c16d23b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:43.478040 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:43.478745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:43.479019 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:43.479285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] PATH_INFO: `/v3/projects/2cb875ffae9841a78e3ff7212dd734f4/users/a623b62f1bd8448f95f7ac8db89d918e/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:43.481872 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cb875ffae9841a78e3ff7212dd734f4, user_id=a623b62f1bd8448f95f7ac8db89d918e, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:43.484628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:43.499670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1381/2802] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 19:58:43 2022] PUT /identity/v3/projects/2cb875ffae9841a78e3ff7212dd734f4/users/a623b62f1bd8448f95f7ac8db89d918e/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:58:51.075037 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26ebb-874e-42f6-bdf6-771ecbeaf6ad None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.075627 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26ebb-874e-42f6-bdf6-771ecbeaf6ad None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.075987 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea26ebb-874e-42f6-bdf6-771ecbeaf6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.119567 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1ea26ebb-874e-42f6-bdf6-771ecbeaf6ad None None] MFA Rules not processed for user `a623b62f1bd8448f95f7ac8db89d918e`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:51.121043 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1ea26ebb-874e-42f6-bdf6-771ecbeaf6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.172733 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:51.233802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1422/2803] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:58:51.254764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:51.257241 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:51.350333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:51.419213 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f191d-fafc-4286-85fd-412d5793cd12 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:51.426248 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:51.426941 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:51.427298 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:51.427664 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:51.515726 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:51.518150 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:51.534193 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1382/2804] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:58:53.010955 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee24-5c8c-4407-a85c-1d57559158a4 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:53.011607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee24-5c8c-4407-a85c-1d57559158a4 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:53.011607 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b154ee24-5c8c-4407-a85c-1d57559158a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:53.037572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b154ee24-5c8c-4407-a85c-1d57559158a4 None None] MFA Rules not processed for user `9ca934350c71421cb644b3a380e96df2`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:58:53.039064 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b154ee24-5c8c-4407-a85c-1d57559158a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:53.048809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1423/2805] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:58:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:59:01.526125 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=46 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:03.031623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=18 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:24.485798 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:24.559661 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689be3a0-977d-45ae-beaa-cc5533570b2f service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:24.564113 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:24.564723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:24.565197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:24.565460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:24.576295 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:24.577835 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13385036-34c8-47f1-bb07-3934e55dc3de service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:24.589667 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1383/2806] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:34.584011 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=26 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:38.343282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:38.343917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:38.343917 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:38.366740 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] MFA Rules not processed for user `02bb08e3dee645e4abe2e6d5e106ea57`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:59:38.368071 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:38.441034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1424/2807] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 19:59:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:59:48.434226 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=25 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:11.092784 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:11.093556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:11.093556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:11.115110 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] MFA Rules not processed for user `4f63b1c74b4b4d34b33bef2b6267ecb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:11.116333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:11.194252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1384/2808] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:00:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:12.087465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:12.099364 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a3c7-5124-4b33-901f-53d80b1cf65d None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:12.103628 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:12.104378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:12.104989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:12.105349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:12.107961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:12.126742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:12.129284 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:12.148390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1425/2809] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:00:16.792096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:16.803036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4e4f2-6f19-4822-9209-501bfa585d09 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:16.806909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:16.807520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:16.807818 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:16.808086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:16.810005 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:16.825482 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:16.826931 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46baec-5aaf-4efb-bbd8-a543a393f48c service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:16.838875 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1385/2810] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:22.139355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:26.833491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=31 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:31.307512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:31.309473 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:31.322092 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01739c8c-9235-41a2-ad7e-849889d6142e service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:31.326916 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:31.327630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:31.328127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:31.328580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:31.330626 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:31.342389 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:31.343882 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:31.353568 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1426/2811] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:41.349180 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=6 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:44.235547 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:44.236385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:44.236385 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:44.258326 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] MFA Rules not processed for user `02bb08e3dee645e4abe2e6d5e106ea57`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:00:44.259555 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:44.268147 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1386/2812] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:00:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:00:44.634332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:44.643165 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedad128-72c3-4a49-bea2-c4cef2af6ac8 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:44.646742 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:44.647404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:44.647698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:44.647964 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:44.649721 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:44.659657 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:44.661150 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:44.669714 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1427/2813] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:45.008135 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:45.009653 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:45.089843 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da176785-1e06-4476-a849-d2d9f446c92f None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:45.098545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:45.099452 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:45.099820 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:45.100197 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:45.102576 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:45.115582 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:45.117312 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:45.127652 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1387/2814] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:47.921813 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:47.937838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae6650-90ea-4a8f-bc23-e6cf9f98dce5 service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:47.944277 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:47.945052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:47.945482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:47.945765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:47.948442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:00:47.963110 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:47.964872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:47.979541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1428/2815] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:55.123342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=35 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:57.974818 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=12 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:38.506087 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:38.508196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:38.535250 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4875e0-01cb-495f-82e0-042fc6112c5f service placement] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:38.544903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:38.545988 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:38.546489 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:38.547059 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:38.549987 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:38.566561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:38.568186 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:38.580982 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1388/2816] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:48.575194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=6 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:57.381476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:57.393824 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc890b00-96e4-45fe-9ad9-9cb1b44a52e8 service nova] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:57.401866 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:57.402509 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:57.402779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:57.403099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:57.406205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:57.428743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:57.433974 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:57.441408 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:57.443791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:57.462345 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1429/2817] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:57.546164 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7af685-b566-4fae-b3c2-13f4489ca5c4 service nova] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:57.551601 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:57.552283 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:57.552613 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:57.552878 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] PATH_INFO: `/v3/users/02bb08e3dee645e4abe2e6d5e106ea57` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:57.554187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=02bb08e3dee645e4abe2e6d5e106ea57)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:57.556216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:57.613940 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02bb08e3dee645e4abe2e6d5e106ea57'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.623801 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02bb08e3dee645e4abe2e6d5e106ea57'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.624745 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02bb08e3dee645e4abe2e6d5e106ea57'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.625657 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02bb08e3dee645e4abe2e6d5e106ea57'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.641546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b49619fa-93b0-4b44-a41a-f1026265d02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02bb08e3dee645e4abe2e6d5e106ea57'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:57.655471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1389/2818] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:57 2022] DELETE /identity/v3/users/02bb08e3dee645e4abe2e6d5e106ea57 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:01:57.885264 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:57.899929 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca48d60-b587-42d3-b512-dc82423858f8 service placement] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:57.977900 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:57.978802 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:57.979140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:57.979557 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] PATH_INFO: `/v3/projects/d7256eb917624a56a2762fcdc3d11b29` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:57.981314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7256eb917624a56a2762fcdc3d11b29)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:57.983447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.045661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7256eb917624a56a2762fcdc3d11b29', 'request_id': 'req-9a0054e3-5112-434e-9de5-b78719cc539d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.053447 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7256eb917624a56a2762fcdc3d11b29', 'request_id': 'req-9a0054e3-5112-434e-9de5-b78719cc539d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.063425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7256eb917624a56a2762fcdc3d11b29', 'request_id': 'req-9a0054e3-5112-434e-9de5-b78719cc539d'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.075606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] The token cache is being invalidate because project d7256eb917624a56a2762fcdc3d11b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:01:58.075908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.076592 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-9a0054e3-5112-434e-9de5-b78719cc539d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:58.078182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1430/2819] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:01:57 2022] DELETE /identity/v3/projects/d7256eb917624a56a2762fcdc3d11b29 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:58.083426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.083821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.084229 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.106846 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.108096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.183806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1390/2820] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:58.193456 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.193741 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.194006 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.216945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.218623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.229196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1431/2821] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:58.233883 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.244703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d414cf6-d854-4e68-aa0f-5ba85f9e373c None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.250911 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.251549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.251826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.252097 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.252765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:01:58.253114 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.254749 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.271401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1391/2822] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:01:58.276533 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.287754 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28221-289a-4fa9-8b92-3e4f8600f7d0 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.292577 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.293236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.293518 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.293789 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.294431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.296378 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.341698 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1432/2823] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:01:58.347413 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.358368 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e1b23-100d-4ed5-85ff-45c30b84b788 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.362948 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:58.428041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.428673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.428957 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.429223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.429889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.431840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.455990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1392/2824] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:01:58.460426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.472052 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227f53c-c69f-478c-9f68-4338237445d9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.477371 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.478009 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.478404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.478561 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.479262 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.480944 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.495513 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1433/2825] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:58.502370 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.513471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4aa045-9267-475a-b365-12538823f757 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.518665 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.519332 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.519611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.519885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.520550 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.522270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b223ee84-5a94-4a88-a62f-34eb8846f945 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.535261 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1393/2826] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:01:58.540349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.559154 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a50af-05e6-45e9-b10a-b559255106ea admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.567248 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.568732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.569349 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.569822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] PATH_INFO: `/v3/projects/89381cd3cca84de38966ba898d8bf920/users/a8abaeb99550447583aafb24276a2285/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.596522 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89381cd3cca84de38966ba898d8bf920, user_id=a8abaeb99550447583aafb24276a2285, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.599302 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.615469 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1434/2827] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:01:58 2022] PUT /identity/v3/projects/89381cd3cca84de38966ba898d8bf920/users/a8abaeb99550447583aafb24276a2285/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:58.626375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.626659 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.626970 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.659897 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] MFA Rules not processed for user `a8abaeb99550447583aafb24276a2285`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:58.661668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.733179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1394/2828] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:58.745390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:58.747275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:58.754270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:58.840242 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8f0fa7-7698-4510-b9aa-0180d2a926e6 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:58.846251 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:58.846961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:58.847119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:58.847417 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:58.856444 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:58.858059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a4c5f6-ab9c-4856-a6f2-9068deeaf3a8 service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:58.868394 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1435/2829] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:01:59.121808 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:59.218238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d1674-2471-4baa-baae-d089a0db2d65 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:59.226344 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:59.227473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:59.227960 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:59.228436 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] PATH_INFO: `/v3/users/a8abaeb99550447583aafb24276a2285` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:59.230707 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8abaeb99550447583aafb24276a2285)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:59.234472 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:59.269502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:59.323461 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8abaeb99550447583aafb24276a2285'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:59.332383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8abaeb99550447583aafb24276a2285'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:59.333069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8abaeb99550447583aafb24276a2285'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:59.333698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8abaeb99550447583aafb24276a2285'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:59.351819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8abaeb99550447583aafb24276a2285'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:01:59.365606 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1395/2830] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:01:59 2022] DELETE /identity/v3/users/a8abaeb99550447583aafb24276a2285 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:01:59.370666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:59.371119 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:59.371493 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:59.402519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:59.404591 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:59.474732 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1436/2831] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:59.485611 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:59.487179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:59.588765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad249a47-d695-4a79-9305-8ef7f067964f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:59.601535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:59.602610 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:59.603106 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:59.603597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:59.622246 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:59.624134 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0506ad9-4d3c-4a23-8af7-283170fb1397 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:59.636627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1396/2832] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:00.031286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.035381 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:00.053243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa661a-e844-4505-aa28-937a96db0f21 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.059729 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.060790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.061272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.061722 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] PATH_INFO: `/v3/projects/89381cd3cca84de38966ba898d8bf920` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.063844 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89381cd3cca84de38966ba898d8bf920)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.067531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.139942 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89381cd3cca84de38966ba898d8bf920', 'request_id': 'req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:00.153173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89381cd3cca84de38966ba898d8bf920', 'request_id': 'req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:00.165731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89381cd3cca84de38966ba898d8bf920', 'request_id': 'req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:00.180530 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] The token cache is being invalidate because project 89381cd3cca84de38966ba898d8bf920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:02:00.180837 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:00.181616 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-b44b3f9d-a7d0-43f0-9416-5f2588a48bc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:00.183257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1437/2833] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:02:00 2022] DELETE /identity/v3/projects/89381cd3cca84de38966ba898d8bf920 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:00.189024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.189469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.189779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.211995 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:00.213322 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:00.291287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1397/2834] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:00.300796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.301233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.301535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.325133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:00.326388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:00.335602 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1438/2835] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:00.340381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.353729 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae1adc-7c54-4fd0-8be4-28bb9b34b2ca None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.359617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.360264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.360546 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.360830 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.361638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:02:00.361927 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.363598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.377160 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1398/2836] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:02:00.384468 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.401640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d3dfa-0d1f-4357-9db9-c754a0ca6d08 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.410508 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.411507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.412114 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.412610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.413833 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.417621 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.426597 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:00.458598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1439/2837] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/projects => generated 393 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:00.463806 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.476177 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123556eb-eecf-4b66-a9ca-0555a40df4a3 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.501095 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:00.545447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.546632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.546914 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.547238 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.548140 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.551836 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.587698 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1399/2838] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/users => generated 297 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:00.591695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.604146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4070a-c16c-45f0-b180-4a27f3692a72 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.610301 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.611005 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.611321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.611639 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.612355 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.614257 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.626501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1440/2839] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:00.631287 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.642907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7908de7-a6c4-431b-98b5-1adc04b3be2b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.648844 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.649520 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.649850 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.650156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.650860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.652594 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6c35e-df5d-4a71-8f42-5099b3a73109 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.665092 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1400/2840] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:00.670332 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.681679 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6589d662-fbc7-4311-950b-5dd67d156f02 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.686656 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.687344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.687613 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.687881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] PATH_INFO: `/v3/projects/6b8cdb93a12d4da2be2bcd16b00ca6c6/users/aededfdb5aea40449825075463255baf/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.705060 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8cdb93a12d4da2be2bcd16b00ca6c6, user_id=aededfdb5aea40449825075463255baf, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.707769 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:00.724605 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1441/2841] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:00 2022] PUT /identity/v3/projects/6b8cdb93a12d4da2be2bcd16b00ca6c6/users/aededfdb5aea40449825075463255baf/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:02:00.737060 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.737390 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.737598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.769399 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] MFA Rules not processed for user `aededfdb5aea40449825075463255baf`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:00.770668 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:00.845518 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1401/2842] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:00.850797 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:00.945084 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d51296-aaf1-4fd2-8fad-a14fb42d2526 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:00.951331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:00.952166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:00.952540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:00.952876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:00.953606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:00.955736 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.002664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1442/2843] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:02:00 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:02:01.004086 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.021573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8560da-5dc7-4497-9add-80a8c2a52719 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.092264 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.093007 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.093352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.093678 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.094481 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.096831 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.121442 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1402/2844] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:02:01 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:02:01.126050 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.138564 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063df0a-e6b1-4e56-9816-a8442b3d24f2 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.144752 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.145448 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.145731 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.146011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.146663 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.148347 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.161411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1443/2845] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:01.166570 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.179226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39dc124-2a76-4f9f-9b2a-f8199005a95f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.185740 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.186484 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.186845 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.187237 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] PATH_INFO: `/v3/projects/6bf961aee18b4ba4aede75c9d029f522/users/8d72c9dee65b4a268299e2b6ee68e6d2/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.206569 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf961aee18b4ba4aede75c9d029f522, user_id=8d72c9dee65b4a268299e2b6ee68e6d2, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.209723 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.221750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:01.230333 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1403/2846] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:01 2022] PUT /identity/v3/projects/6bf961aee18b4ba4aede75c9d029f522/users/8d72c9dee65b4a268299e2b6ee68e6d2/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:02:01.234845 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.246033 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb1f32-04b6-4f60-882c-30012f2b10d0 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.319452 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.320476 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.321048 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.321523 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.322640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.325856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b3d73-336e-4733-9945-73275a35592e admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.346664 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1444/2847] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:01.351578 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.363535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbf70c-6a43-4913-87f4-504ae93cf1a5 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.369783 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.370597 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] REQUEST_METHOD: `PUT` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.370990 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.371375 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] PATH_INFO: `/v3/projects/6bf961aee18b4ba4aede75c9d029f522/users/8d72c9dee65b4a268299e2b6ee68e6d2/roles/8b79e1e620da45a9bc66fc22b41b2a58` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.374041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf961aee18b4ba4aede75c9d029f522, user_id=8d72c9dee65b4a268299e2b6ee68e6d2, role_id=8b79e1e620da45a9bc66fc22b41b2a58)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.377223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.393527 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1404/2848] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:01 2022] PUT /identity/v3/projects/6bf961aee18b4ba4aede75c9d029f522/users/8d72c9dee65b4a268299e2b6ee68e6d2/roles/8b79e1e620da45a9bc66fc22b41b2a58 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:01.412177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.412648 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.413162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.465367 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] MFA Rules not processed for user `8d72c9dee65b4a268299e2b6ee68e6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:01.467446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:01.483333 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:01.564995 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1445/2849] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:01.577082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.580433 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:01.731188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7977099d-f1b2-419a-8313-de9bb900ac64 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.735222 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.735853 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.736129 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.736397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.747434 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.749223 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.761365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1405/2850] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:01 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:01.818548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:01.820272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:01.833147 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece4952-359d-4370-a36f-f5616ba168c2 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:01.837272 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:01.837913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:01.838187 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:01.838491 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:01.905177 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:01.906647 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d040c70-4933-46f3-9d37-c3e41414fb06 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:01.918019 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1446/2851] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:01 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:05.858418 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:05.935187 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62e8dc-91e2-44a3-b694-66ab078787ee service cinder] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:05.940359 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:05.941265 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:05.941586 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:05.941860 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] PATH_INFO: `/v3/users/aededfdb5aea40449825075463255baf` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:05.943204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aededfdb5aea40449825075463255baf)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:05.945308 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:06.007528 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aededfdb5aea40449825075463255baf'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.018041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aededfdb5aea40449825075463255baf'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.018979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aededfdb5aea40449825075463255baf'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.019765 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aededfdb5aea40449825075463255baf'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.035145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:06.037339 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aededfdb5aea40449825075463255baf'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.055420 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1406/2852] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:02:05 2022] DELETE /identity/v3/users/aededfdb5aea40449825075463255baf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:02:06.060918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:06.061506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:06.061506 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:06.085765 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:06.087173 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:06.165387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1447/2853] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:06.177382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:06.179173 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:06.268508 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83115113-b6a2-4cea-899c-e86d6c11fbd0 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:06.272842 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:06.273511 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:06.273802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:06.274056 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:06.285320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:06.286774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:06.298194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1407/2854] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:02:06.623426 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:06.626045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:06.648169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08910654-f3fc-4be0-b716-577ac0094e17 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:06.658655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:06.659755 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:06.660227 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:06.660688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] PATH_INFO: `/v3/projects/6b8cdb93a12d4da2be2bcd16b00ca6c6` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:06.662868 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8cdb93a12d4da2be2bcd16b00ca6c6)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:06.666595 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:06.724482 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:06.737975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8cdb93a12d4da2be2bcd16b00ca6c6', 'request_id': 'req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.746540 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8cdb93a12d4da2be2bcd16b00ca6c6', 'request_id': 'req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.756684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8cdb93a12d4da2be2bcd16b00ca6c6', 'request_id': 'req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.774504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] The token cache is being invalidate because project 6b8cdb93a12d4da2be2bcd16b00ca6c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:02:06.775014 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.776159 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.778889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1448/2855] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:02:06 2022] DELETE /identity/v3/projects/6b8cdb93a12d4da2be2bcd16b00ca6c6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:06.783137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:06.794123 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999e480-8ee2-46a3-83f0-4cf3634decc2 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:06.883819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:06.884554 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:06.884755 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:06.885228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] PATH_INFO: `/v3/users/8d72c9dee65b4a268299e2b6ee68e6d2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:06.886538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d72c9dee65b4a268299e2b6ee68e6d2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:06.888615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:06.954840 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d72c9dee65b4a268299e2b6ee68e6d2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.964488 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d72c9dee65b4a268299e2b6ee68e6d2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.965400 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d72c9dee65b4a268299e2b6ee68e6d2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.966043 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d72c9dee65b4a268299e2b6ee68e6d2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:06.985638 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-da7f608a-bbbc-40ce-a4f3-f85a74c8fc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d72c9dee65b4a268299e2b6ee68e6d2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.009357 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1408/2856] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:02:06 2022] DELETE /identity/v3/users/8d72c9dee65b4a268299e2b6ee68e6d2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:07.364245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:07.366502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:07.389972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0728fdb3-93e3-4977-8f62-6ad4ec8b54ac admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:07.481526 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:07.482458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.482839 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.483253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] PATH_INFO: `/v3/projects/6bf961aee18b4ba4aede75c9d029f522` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.484735 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf961aee18b4ba4aede75c9d029f522)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:07.487218 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:07.560245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf961aee18b4ba4aede75c9d029f522', 'request_id': 'req-4a1f888c-2f81-4b62-8ede-c8574b8763e2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.572133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf961aee18b4ba4aede75c9d029f522', 'request_id': 'req-4a1f888c-2f81-4b62-8ede-c8574b8763e2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.583695 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf961aee18b4ba4aede75c9d029f522', 'request_id': 'req-4a1f888c-2f81-4b62-8ede-c8574b8763e2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.602386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] The token cache is being invalidate because project 6bf961aee18b4ba4aede75c9d029f522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:02:07.602883 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.603943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-4a1f888c-2f81-4b62-8ede-c8574b8763e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:02:07.606336 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1449/2857] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:02:07 2022] DELETE /identity/v3/projects/6bf961aee18b4ba4aede75c9d029f522 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:02:07.613396 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.613743 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.613929 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.637734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:07.639036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:07.644447 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:07.718054 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1409/2858] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:02:07.727046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.727329 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.727606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.757100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:07.759124 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:07.770667 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1450/2859] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:07.776486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:07.788486 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3414ba-b988-4f3d-a577-56f042c17022 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:07.794812 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:07.795641 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.796017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.796426 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.797299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:02:07.797793 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:07.799976 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:07.816256 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1410/2860] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:02:07.822314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:07.832717 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3051b7-d669-44a8-85b5-e05b2e34b491 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:07.838862 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:07.839507 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.839785 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.840051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.840694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:07.842861 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:07.877590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1451/2861] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:02:07 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:07.882849 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:07.898670 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd1283-75c1-4d14-a3f3-cdf78caeb40d admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:07.980717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401db792-122f-4e36-a393-018ced760357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:07.981634 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db792-122f-4e36-a393-018ced760357 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:07.982076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db792-122f-4e36-a393-018ced760357 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:07.982627 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401db792-122f-4e36-a393-018ced760357 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:07.983542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401db792-122f-4e36-a393-018ced760357 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:07.985969 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401db792-122f-4e36-a393-018ced760357 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.015695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1411/2862] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:02:07 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:02:08.020975 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.041146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfcfc-11e9-4a03-9502-f89d7000e1ac admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.047576 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.048360 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.049086 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.049427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.050182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.052069 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.052393 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db371-bc0f-4b55-92db-963c6ee0ed08 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.052819 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db371-bc0f-4b55-92db-963c6ee0ed08 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.053235 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76db371-bc0f-4b55-92db-963c6ee0ed08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.064170 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1452/2863] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:02:08.068670 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.078200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-d76db371-bc0f-4b55-92db-963c6ee0ed08 None None] MFA Rules not processed for user `4f63b1c74b4b4d34b33bef2b6267ecb2`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:08.079615 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26f419e-f05b-4fdd-847f-c4262bb602ea admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.079826 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d76db371-bc0f-4b55-92db-963c6ee0ed08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:08.087696 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.088684 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.089203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.089646 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.090889 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.093886 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.117267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1453/2864] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:02:08.119743 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.138175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd218c-4a9c-49a2-86b1-fbd2ccb41a02 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.145157 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.146107 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.146484 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.146864 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] PATH_INFO: `/v3/projects/241fa7029c7d46518c2e36983545cd5c/users/e240680bbf2c4f02a1429b335261de3d/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.148077 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1412/2865] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:02:08.150934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:08.178099 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=241fa7029c7d46518c2e36983545cd5c, user_id=e240680bbf2c4f02a1429b335261de3d, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.182914 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.204106 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1454/2866] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:02:08 2022] PUT /identity/v3/projects/241fa7029c7d46518c2e36983545cd5c/users/e240680bbf2c4f02a1429b335261de3d/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:02:08.210636 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb258-021a-4322-8026-1e78394d3e6d None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.211100 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb258-021a-4322-8026-1e78394d3e6d None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.211545 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeacb258-021a-4322-8026-1e78394d3e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.237880 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-aeacb258-021a-4322-8026-1e78394d3e6d None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:08.239471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-aeacb258-021a-4322-8026-1e78394d3e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:08.290476 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:08.356423 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1413/2867] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:08.370948 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:08.373881 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:08.455205 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227ccd1-4c1a-442d-a58b-001d402aca14 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:08.459253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:08.459906 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:08.460185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:08.460457 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:08.471454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:08.473193 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:08.496961 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1455/2868] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:09.469979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929c2b-0eee-40b7-9348-5aeb77a42792 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:09.470907 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929c2b-0eee-40b7-9348-5aeb77a42792 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:09.471417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd929c2b-0eee-40b7-9348-5aeb77a42792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:09.522604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-bd929c2b-0eee-40b7-9348-5aeb77a42792 None None] MFA Rules not processed for user `e240680bbf2c4f02a1429b335261de3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:09.525220 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bd929c2b-0eee-40b7-9348-5aeb77a42792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:09.660179 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1414/2869] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:09.678263 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:09.680951 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:09.814321 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a539c-98e6-4cb3-9a51-05d0a6741f5b service neutron] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:09.820245 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:09.821584 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:09.822223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:09.822760 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:09.842013 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:09.847162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:09.863454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1456/2870] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:11.466416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acdbed-8673-47aa-b9a7-34a1c0deb08e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:11.467254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acdbed-8673-47aa-b9a7-34a1c0deb08e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:11.467817 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acdbed-8673-47aa-b9a7-34a1c0deb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:11.512521 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-91acdbed-8673-47aa-b9a7-34a1c0deb08e None None] MFA Rules not processed for user `e240680bbf2c4f02a1429b335261de3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:11.514364 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-91acdbed-8673-47aa-b9a7-34a1c0deb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:11.525604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1415/2871] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:16.538548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:16.540632 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:16.644822 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47470de7-452b-412a-ba9d-eff45e1e555c service cinder] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:16.651136 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '844040a2fb60437a9188ccb5293d404b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:16.652070 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:16.652541 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:16.653046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:16.655956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:16.672519 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:16.674807 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e42c3-e801-4932-bc37-615ba1a15ecc service nova] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:16.686933 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1457/2872] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:02:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:02:21.501342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=48 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:26.681934 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:30.937270 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:30.938036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:30.938036 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:30.967335 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] MFA Rules not processed for user `e240680bbf2c4f02a1429b335261de3d`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:30.969299 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:30.982301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1416/2873] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:02:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:40.960276 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:51.903353 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ee4e-f4a9-45cf-8dda-8c9abc5af479 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:51.904021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ee4e-f4a9-45cf-8dda-8c9abc5af479 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:51.904021 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243ee4e-f4a9-45cf-8dda-8c9abc5af479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:51.972852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3243ee4e-f4a9-45cf-8dda-8c9abc5af479 None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:02:51.974113 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3243ee4e-f4a9-45cf-8dda-8c9abc5af479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:52.043250 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1458/2874] 10.222.0.18 () {58 vars in 1009 bytes} [Tue Sep 13 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:02:52.097567 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:02:52.100841 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:02:52.214886 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d5815-468f-40ab-aee9-7a6a1945914a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:02:52.220979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:02:52.221603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:02:52.221881 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:02:52.222152 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:02:52.231750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:02:52.233397 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:02:52.243124 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1417/2875] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 20:02:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:03:02.039453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=44 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:02.238774 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=41 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:07.462852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:07.463376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:07.463376 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:07.484449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] MFA Rules not processed for user `be2777d5f726425e9d2c144b5d98f715`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:07.485768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:07.493801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1459/2876] 10.222.0.18 () {58 vars in 967 bytes} [Tue Sep 13 20:03:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:07.506535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:07.517585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5961b2-7054-439b-86ec-d2367d5bdbf9 service swift] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:07.523738 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae58835c32e4a8f9e06f0a4c65ce26b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:07.524361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:07.524626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:07.524981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:07.537777 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:07.539392 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:07.550010 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1418/2877] 10.222.0.18 () {60 vars in 1384 bytes} [Tue Sep 13 20:03:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:03:09.473622 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:09.475281 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:09.536945 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:09.548566 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c8109-4beb-4c99-b8b5-4f2f0c23d701 service swift] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:09.554354 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c395d5-67cb-43f5-a925-d739eae89c7e None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:09.554483 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:09.555014 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:09.555285 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:09.555603 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] PATH_INFO: `/v3/users/e240680bbf2c4f02a1429b335261de3d` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:09.556674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e240680bbf2c4f02a1429b335261de3d)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:09.558605 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:09.562876 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:09.563611 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:09.563893 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:09.564168 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:09.566034 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:09.578169 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:09.579751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:09.590640 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1460/2878] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:09.609301 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e240680bbf2c4f02a1429b335261de3d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.618681 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e240680bbf2c4f02a1429b335261de3d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.619327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e240680bbf2c4f02a1429b335261de3d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.619923 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e240680bbf2c4f02a1429b335261de3d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.632232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-d9a5a445-5e22-4ac4-9a5c-83f0724bc7eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e240680bbf2c4f02a1429b335261de3d'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.645185 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1419/2879] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:09 2022] DELETE /identity/v3/users/e240680bbf2c4f02a1429b335261de3d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:09.829521 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:09.841087 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873132a-ac4a-4098-867e-35f1fdc32ba9 service placement] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:09.912387 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:09.913076 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:09.913356 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:09.913630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] PATH_INFO: `/v3/projects/241fa7029c7d46518c2e36983545cd5c` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:09.914790 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=241fa7029c7d46518c2e36983545cd5c)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:09.916750 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:09.967303 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '241fa7029c7d46518c2e36983545cd5c', 'request_id': 'req-8353f375-ff1a-4cf2-8fab-421a5950a5b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.974442 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '241fa7029c7d46518c2e36983545cd5c', 'request_id': 'req-8353f375-ff1a-4cf2-8fab-421a5950a5b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.984167 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '241fa7029c7d46518c2e36983545cd5c', 'request_id': 'req-8353f375-ff1a-4cf2-8fab-421a5950a5b6'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.995359 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] The token cache is being invalidate because project 241fa7029c7d46518c2e36983545cd5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:09.995606 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.996278 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8353f375-ff1a-4cf2-8fab-421a5950a5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:09.997950 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1461/2880] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:09 2022] DELETE /identity/v3/projects/241fa7029c7d46518c2e36983545cd5c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:03:10.003188 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.003473 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.003746 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.024809 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:10.026080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:10.092449 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1420/2881] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:10.100608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.100909 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.101209 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.121863 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:10.123089 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:10.133538 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1462/2882] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:10.138717 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.154863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513b5d-dd4e-4c52-b152-2db001fe68fb None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.161537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.162136 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.162411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.162673 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.163342 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:03:10.163639 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.165252 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.178695 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1421/2883] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:03:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:03:10.183445 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.193977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856d9e3-2f65-432c-83df-0235d6adbfe1 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.200618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.201307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.201585 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.201852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.202501 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.204545 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.209609 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:10.238488 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1463/2884] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:03:10 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:10.243602 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.258991 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775121c-bbce-465c-bbea-66b3d6167919 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.311132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:10.327415 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.328042 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.328363 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.328630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.329303 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.331428 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.354815 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1422/2885] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:03:10 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:10.359133 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.370256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28808cf3-3cda-40c8-9f11-00d127a824a3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.375688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.376313 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.376586 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.376859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.377534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.379175 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.390764 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1464/2886] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:10.394894 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.405127 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b7534-fe17-4814-a262-2e745cf6f8fc admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.410073 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.410647 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.410928 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.411215 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.411833 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.413455 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49e41d-d739-402c-b7b5-1b3f570f10d6 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.425693 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1423/2887] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:10.430504 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.441388 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17f73-f9e8-40ea-8348-60d88b9ef393 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.448551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.449293 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.449572 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.449843 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] PATH_INFO: `/v3/projects/10ca0ddf9dd6438784290d12824669bc/users/c431e346666c4c55ac40145eac754c24/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.466666 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10ca0ddf9dd6438784290d12824669bc, user_id=c431e346666c4c55ac40145eac754c24, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.469453 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.485918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1465/2888] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:03:10 2022] PUT /identity/v3/projects/10ca0ddf9dd6438784290d12824669bc/users/c431e346666c4c55ac40145eac754c24/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:10.500460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.500769 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.501076 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.531485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] MFA Rules not processed for user `c431e346666c4c55ac40145eac754c24`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:10.532697 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:10.600355 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1424/2889] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:10.611809 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.613565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:10.677394 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61ae99-f223-4ea7-9a9a-a320082924e4 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.681768 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.682386 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.682645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.682911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.692431 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.694233 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e84792d-3451-4e86-aab5-100f2ed11cda service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.707623 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1466/2890] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:13.323944 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:13.394242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905e0af-8761-4590-aae1-ea908b23579a None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:13.399362 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:13.399996 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:13.400249 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:13.400534 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] PATH_INFO: `/v3/users/c431e346666c4c55ac40145eac754c24` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:13.401747 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c431e346666c4c55ac40145eac754c24)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:13.403676 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:13.462411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c431e346666c4c55ac40145eac754c24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:13.470858 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c431e346666c4c55ac40145eac754c24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:13.471580 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c431e346666c4c55ac40145eac754c24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:13.472242 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c431e346666c4c55ac40145eac754c24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:13.474192 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:13.486200 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c431e346666c4c55ac40145eac754c24'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:13.498251 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1425/2891] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:13 2022] DELETE /identity/v3/users/c431e346666c4c55ac40145eac754c24 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:13.502925 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:13.503432 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:13.503556 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:13.522982 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:13.524182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:13.601146 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1467/2892] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:13.611236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:13.612660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:13.685354 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7c1c3-c664-419c-853d-4e8ee2f1fd1f admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:13.689742 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:13.690365 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:13.690640 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:13.690909 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:13.701239 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:13.702689 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c106e-f306-447c-ac00-095f0eb74d72 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:13.713132 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1426/2893] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:14.001196 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.002697 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:14.016145 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3ec338-a757-47d7-92f0-d95c7ff35b5a None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.022856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.023553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.023957 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.024992 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] PATH_INFO: `/v3/projects/10ca0ddf9dd6438784290d12824669bc` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.026256 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10ca0ddf9dd6438784290d12824669bc)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.028305 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.084398 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:14.085857 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10ca0ddf9dd6438784290d12824669bc', 'request_id': 'req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:14.094918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10ca0ddf9dd6438784290d12824669bc', 'request_id': 'req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:14.104779 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10ca0ddf9dd6438784290d12824669bc', 'request_id': 'req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:14.116225 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] The token cache is being invalidate because project 10ca0ddf9dd6438784290d12824669bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:14.116446 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:14.117127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-c1ef52af-21c6-4095-8a1e-3cea6140cdf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:14.118645 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1468/2894] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:13 2022] DELETE /identity/v3/projects/10ca0ddf9dd6438784290d12824669bc => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:14.123764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.124093 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.124367 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.146829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:14.148041 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:14.218705 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1427/2895] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:14.230104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.230454 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.230826 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.253252 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:14.254465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:14.262852 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1469/2896] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:14.267422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.277714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4a169-cfb7-42b1-a0f3-b6830b9015cc None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.283023 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.283622 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.283896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.284162 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.284920 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:03:14.285243 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.286856 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.298585 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1428/2897] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:03:14.303575 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.313911 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f59c8-3ee7-4a60-8219-d473570953fa None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.319244 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.319850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.320129 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.320401 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.321077 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.322986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.359307 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1470/2898] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:03:14 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:14.365896 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.380336 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f8f6b-8682-4189-bbcd-d66828360cac admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.449719 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.450340 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.450617 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.450885 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.451523 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.453500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.464288 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:14.486598 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1429/2899] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:03:14 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:14.492878 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.506236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d1020-6936-4191-a7ba-1ca6afe616c1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.511346 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.511945 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.512217 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.512485 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.513171 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.514758 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.525373 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1471/2900] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:14.529401 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.539537 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e84374-16fd-41d7-be1a-65e279e794ad admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.544445 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.545080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.545379 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.545632 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.546273 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.547932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da11223-8845-483b-8744-7f8cd510fa30 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.558967 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1430/2901] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:14.563253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.573534 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067ae76-92ae-4a24-9de4-7251142c88a8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.578323 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.578977 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.579270 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.579527 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] PATH_INFO: `/v3/projects/3319879f5a4e4641ae2815149b5ea580/users/7b8b979523fa4edda19d51da308c67b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.594104 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3319879f5a4e4641ae2815149b5ea580, user_id=7b8b979523fa4edda19d51da308c67b6, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.597059 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.612976 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1472/2902] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:03:14 2022] PUT /identity/v3/projects/3319879f5a4e4641ae2815149b5ea580/users/7b8b979523fa4edda19d51da308c67b6/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:14.623245 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.623516 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.623790 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.654649 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] MFA Rules not processed for user `7b8b979523fa4edda19d51da308c67b6`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:14.655780 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:14.726262 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1431/2903] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:03:14.738289 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.740314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:14.801018 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc7b39-d783-47f6-85a1-ed8be5ad776a admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.805498 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.806127 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.806372 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.806652 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.816370 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.818047 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f081d-cd6b-4b9a-abb5-7d8c126bc3d8 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:14.827465 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1473/2904] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:14.852382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:14.935888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284b531-1a8d-4060-a3f7-2661a79be43e None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:14.943102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:14.943102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:14.943102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:14.943102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] PATH_INFO: `/v3/users/7b8b979523fa4edda19d51da308c67b6` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:14.943722 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8b979523fa4edda19d51da308c67b6)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:14.945734 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:15.002320 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8b979523fa4edda19d51da308c67b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.013750 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8b979523fa4edda19d51da308c67b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.014615 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8b979523fa4edda19d51da308c67b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.015648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8b979523fa4edda19d51da308c67b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.034082 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8b979523fa4edda19d51da308c67b6'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.036137 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:15.048538 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1432/2905] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:14 2022] DELETE /identity/v3/users/7b8b979523fa4edda19d51da308c67b6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:15.053688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.053973 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.054269 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.074598 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:15.075841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:15.146922 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1474/2906] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:15.159274 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:15.160737 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:15.223458 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baacd14-8876-4540-a361-5c6aafa0e88c admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:15.228080 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:15.228714 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.229032 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.229323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.240762 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:15.242504 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3940f7-5647-4e49-86bc-e4bbcb98f8ba service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:15.255232 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1433/2907] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:15.563918 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:15.565496 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:15.577801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8293df-72e3-462c-b1d3-d55cefc7f6bb None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:15.583223 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:15.583892 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.584149 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.584458 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] PATH_INFO: `/v3/projects/3319879f5a4e4641ae2815149b5ea580` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.585610 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3319879f5a4e4641ae2815149b5ea580)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:15.587531 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:15.599240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:15.644123 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3319879f5a4e4641ae2815149b5ea580', 'request_id': 'req-51f39254-31db-4fb0-94aa-1fe1cbe38a93'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.650989 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3319879f5a4e4641ae2815149b5ea580', 'request_id': 'req-51f39254-31db-4fb0-94aa-1fe1cbe38a93'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.661801 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3319879f5a4e4641ae2815149b5ea580', 'request_id': 'req-51f39254-31db-4fb0-94aa-1fe1cbe38a93'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.673759 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] The token cache is being invalidate because project 3319879f5a4e4641ae2815149b5ea580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:15.674045 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.674709 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-51f39254-31db-4fb0-94aa-1fe1cbe38a93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:15.676253 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1475/2908] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:15 2022] DELETE /identity/v3/projects/3319879f5a4e4641ae2815149b5ea580 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:15.681862 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.682352 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.682460 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.703517 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:15.704899 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:15.765975 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1434/2909] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:15.773998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.774282 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.774565 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.801956 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:15.803796 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:15.816601 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1476/2910] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:03:15.821038 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:15.831429 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b89a9c-2b7b-4058-bf3e-3e703b16f719 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:15.837502 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:15.838107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.838382 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.838648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] PATH_INFO: `/v3/domains` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.839323 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:03:15.839626 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:15.841240 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:15.853510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1435/2911] 10.222.0.18 () {58 vars in 1153 bytes} [Tue Sep 13 20:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:03:15.857920 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:15.867550 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630f848-dc36-472c-989f-3b615a786c50 None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:15.875399 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:15.876151 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:15.876425 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:15.876693 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:15.877366 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:15.879340 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:15.917121 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1477/2912] 10.222.0.18 () {60 vars in 1153 bytes} [Tue Sep 13 20:03:15 2022] POST /identity/v3/projects => generated 397 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:15.921604 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:15.932577 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc976e9-8319-4f46-81a6-b9ebd51ca53b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:16.003171 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:16.003915 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.004194 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.004491 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] PATH_INFO: `/v3/users` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.005156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:16.007085 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:16.036272 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1436/2913] 10.222.0.18 () {60 vars in 1144 bytes} [Tue Sep 13 20:03:15 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:03:16.037571 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:16.048774 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bf94b-abd2-4de9-9c34-b25d0cbed5fc admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:16.054859 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:16.055512 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.055786 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.056051 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.056694 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:16.058352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:16.070529 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1478/2914] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:16.075104 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:16.086111 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ffe8c9-3aeb-4937-ad6d-d11e4218f596 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:16.090961 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:16.091648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.091980 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.092254 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] PATH_INFO: `/v3/roles` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.092932 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:16.094703 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312857fc-df03-4ca8-b31f-6b9afb78355d admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:16.100500 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:16.106767 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1437/2915] 10.222.0.18 () {58 vars in 1122 bytes} [Tue Sep 13 20:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:03:16.111283 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:16.121799 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac3cf9-dd6e-496f-992c-a029564886e9 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:16.126936 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:16.127600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.127872 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.128140 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] PATH_INFO: `/v3/projects/af0e702cc25f4980bb0b87ec53e3ba13/users/2d5fb05ced6740b8a88c0a2e239b8f81/roles/7e3d833e6ff343faa2a2b56eadd5cc59` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.145393 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af0e702cc25f4980bb0b87ec53e3ba13, user_id=2d5fb05ced6740b8a88c0a2e239b8f81, role_id=7e3d833e6ff343faa2a2b56eadd5cc59)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:16.148042 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:16.162100 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1479/2916] 10.222.0.18 () {60 vars in 1483 bytes} [Tue Sep 13 20:03:16 2022] PUT /identity/v3/projects/af0e702cc25f4980bb0b87ec53e3ba13/users/2d5fb05ced6740b8a88c0a2e239b8f81/roles/7e3d833e6ff343faa2a2b56eadd5cc59 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:16.174214 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] REQUEST_METHOD: `POST` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.174497 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.174764 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.203651 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.auth.core [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] MFA Rules not processed for user `2d5fb05ced6740b8a88c0a2e239b8f81`. Rule list: `[]` (Enabled: `True`). {{(pid=93091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:16.204922 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:16.275981 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1438/2917] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:16.288310 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:16.291141 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:16.373817 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59808090-5448-4945-b2f8-6168d0ed28e1 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:16.378382 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e306d2cfa2ce47b39520ccaf3daa6a10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:16.379943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:16.379943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:16.379943 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:16.393704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:16.395162 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efcf179-ae9d-46c2-aefd-18f380e00c14 service cinder] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:16.404998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1480/2918] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:16 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:24.833226 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:24.904865 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9430975-b0d2-4c60-ae8f-b806d8a654d9 None None] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:24.910156 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:24.910863 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:24.911216 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:24.911417 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] PATH_INFO: `/v3/users/2d5fb05ced6740b8a88c0a2e239b8f81` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:24.912573 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d5fb05ced6740b8a88c0a2e239b8f81)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:24.914549 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:24.969888 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d5fb05ced6740b8a88c0a2e239b8f81'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:24.979145 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d5fb05ced6740b8a88c0a2e239b8f81'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:24.979949 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d5fb05ced6740b8a88c0a2e239b8f81'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:24.980556 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d5fb05ced6740b8a88c0a2e239b8f81'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:24.994049 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d5fb05ced6740b8a88c0a2e239b8f81'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.007608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1439/2919] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:24 2022] DELETE /identity/v3/users/2d5fb05ced6740b8a88c0a2e239b8f81 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:25.012325 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] REQUEST_METHOD: `POST` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:25.012850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:25.012986 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:25.033963 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.auth.core [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] MFA Rules not processed for user `cc92c723f30741629d9940fdc108b101`. Rule list: `[]` (Enabled: `True`). {{(pid=93090) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:03:25.035240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:25.068515 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:25.100011 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1481/2920] 10.222.0.18 () {58 vars in 958 bytes} [Tue Sep 13 20:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:03:25.112107 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:25.113821 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:25.128000 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:25.181979 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074fadc4-0f1c-4ede-ac10-28defabc2a08 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:25.185736 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:25.186402 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:25.186674 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:25.186943 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:25.197116 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:25.203781 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:25.215608 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1440/2921] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:03:25.494144 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:25.495690 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:25.513215 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e409bb5-f718-4b1a-9615-20857b24becd None None] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:25.519035 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:25.519823 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:25.520210 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:25.520385 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] PATH_INFO: `/v3/projects/af0e702cc25f4980bb0b87ec53e3ba13` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:25.521549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af0e702cc25f4980bb0b87ec53e3ba13)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:25.523569 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:25.587404 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af0e702cc25f4980bb0b87ec53e3ba13', 'request_id': 'req-adc91172-4214-41f8-9092-a05c62693da3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.597038 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af0e702cc25f4980bb0b87ec53e3ba13', 'request_id': 'req-adc91172-4214-41f8-9092-a05c62693da3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.608182 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af0e702cc25f4980bb0b87ec53e3ba13', 'request_id': 'req-adc91172-4214-41f8-9092-a05c62693da3'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.622297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] The token cache is being invalidate because project af0e702cc25f4980bb0b87ec53e3ba13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:25.622608 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.623704 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:25.627593 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1482/2922] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:25 2022] DELETE /identity/v3/projects/af0e702cc25f4980bb0b87ec53e3ba13 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:31.594925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:31.596471 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:31.614800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da643c-9e62-46c3-8178-cf1fa97b4630 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:31.673325 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:31.673889 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:31.674143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:31.674411 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] PATH_INFO: `/v3/users/9ca934350c71421cb644b3a380e96df2` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:31.675548 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ca934350c71421cb644b3a380e96df2)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:31.677910 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:31.727519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ca934350c71421cb644b3a380e96df2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:31.736779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ca934350c71421cb644b3a380e96df2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:31.737437 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ca934350c71421cb644b3a380e96df2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:31.738044 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ca934350c71421cb644b3a380e96df2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:31.750103 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ca934350c71421cb644b3a380e96df2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:31.764422 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1441/2923] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:31 2022] DELETE /identity/v3/users/9ca934350c71421cb644b3a380e96df2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:31.966778 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:31.968580 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:31.982630 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91172-4214-41f8-9092-a05c62693da3 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:32.053751 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:32.054411 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:32.054692 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:32.054971 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] PATH_INFO: `/v3/projects/838511c93b924f118bcfc03ecfd870b2` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:32.056178 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838511c93b924f118bcfc03ecfd870b2)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:32.058134 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:32.117449 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838511c93b924f118bcfc03ecfd870b2', 'request_id': 'req-8a3ba938-c309-4b5c-8615-1c9aeb844447'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.124054 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838511c93b924f118bcfc03ecfd870b2', 'request_id': 'req-8a3ba938-c309-4b5c-8615-1c9aeb844447'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.132749 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838511c93b924f118bcfc03ecfd870b2', 'request_id': 'req-8a3ba938-c309-4b5c-8615-1c9aeb844447'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.134775 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:32.145416 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] The token cache is being invalidate because project 838511c93b924f118bcfc03ecfd870b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:32.145738 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.146549 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.148139 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1483/2924] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:31 2022] DELETE /identity/v3/projects/838511c93b924f118bcfc03ecfd870b2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:32.152360 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:32.153903 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:32.167479 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0377d34-a23b-4034-b78e-1f3a8513032b admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:32.204204 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:32.239630 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:32.240182 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:32.240469 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:32.240725 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] PATH_INFO: `/v3/users/a623b62f1bd8448f95f7ac8db89d918e` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:32.241925 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a623b62f1bd8448f95f7ac8db89d918e)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:32.243847 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:32.293143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a623b62f1bd8448f95f7ac8db89d918e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.302587 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a623b62f1bd8448f95f7ac8db89d918e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.303327 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a623b62f1bd8448f95f7ac8db89d918e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.304017 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a623b62f1bd8448f95f7ac8db89d918e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.316416 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a623b62f1bd8448f95f7ac8db89d918e'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.329309 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1442/2925] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:03:32 2022] DELETE /identity/v3/users/a623b62f1bd8448f95f7ac8db89d918e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:32.516555 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:32.518228 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:03:32.532972 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3ba938-c309-4b5c-8615-1c9aeb844447 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:32.607203 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:32.607998 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:32.608286 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:32.608551 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] PATH_INFO: `/v3/projects/2cb875ffae9841a78e3ff7212dd734f4` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:32.609727 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cb875ffae9841a78e3ff7212dd734f4)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:32.611665 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:32.666020 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cb875ffae9841a78e3ff7212dd734f4', 'request_id': 'req-6fff1525-cd3a-43c5-9451-b3571cdff757'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.675672 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cb875ffae9841a78e3ff7212dd734f4', 'request_id': 'req-6fff1525-cd3a-43c5-9451-b3571cdff757'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.684908 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cb875ffae9841a78e3ff7212dd734f4', 'request_id': 'req-6fff1525-cd3a-43c5-9451-b3571cdff757'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.696028 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] The token cache is being invalidate because project 2cb875ffae9841a78e3ff7212dd734f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93090) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:03:32.696314 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.700655 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:03:32.702143 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1484/2926] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:03:32 2022] DELETE /identity/v3/projects/2cb875ffae9841a78e3ff7212dd734f4 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:03:42.325624 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:42.692819 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=54,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:10.512157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:10.514196 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:10.583381 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e2b1-fecd-4165-a4ce-bccb44eec057 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:10.633069 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:10.633679 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:10.633952 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:10.634228 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:10.636157 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:10.712561 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:10.714540 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:10.727024 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1443/2927] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:14.686166 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:14.687762 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:14.760548 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff1525-cd3a-43c5-9451-b3571cdff757 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:14.765590 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:14.766267 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:14.766552 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:14.766761 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:14.768661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:14.780618 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:14.782427 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:14.792441 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1485/2928] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:20.722315 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=68 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:24.390143 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:24.391628 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:24.402802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398d94e-04ae-4b03-89d8-c053cf877c28 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:24.406779 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ce3f12892d141c88071095f6fe6506d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:24.407383 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] REQUEST_METHOD: `GET` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:24.407660 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:24.407935 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:24.409759 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:24.472684 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:24.474166 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:24.484168 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1444/2929] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:24.788236 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=32 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:31.047297 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:31.048781 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:31.064788 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36824a1c-918f-4f7f-9fc3-50239b6d737b service placement] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:31.069913 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:31.070559 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:31.070841 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:31.071109 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] PATH_INFO: `/v3/users/4f63b1c74b4b4d34b33bef2b6267ecb2` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:31.072296 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f63b1c74b4b4d34b33bef2b6267ecb2)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:31.074258 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:31.130800 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f63b1c74b4b4d34b33bef2b6267ecb2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.138462 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f63b1c74b4b4d34b33bef2b6267ecb2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.139479 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f63b1c74b4b4d34b33bef2b6267ecb2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.140344 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f63b1c74b4b4d34b33bef2b6267ecb2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.149838 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f63b1c74b4b4d34b33bef2b6267ecb2'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.172240 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1486/2930] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:04:31 2022] DELETE /identity/v3/users/4f63b1c74b4b4d34b33bef2b6267ecb2 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:04:31.380648 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:31.391800 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b85e9-cd07-4f53-a3a4-e6631d905eb8 service neutron] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:31.458600 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:31.459236 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:31.459535 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:31.459802 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] PATH_INFO: `/v3/projects/245af7b7ed61405ba444cddd26fc69fd` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:31.460930 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=245af7b7ed61405ba444cddd26fc69fd)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:31.462834 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:31.511519 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '245af7b7ed61405ba444cddd26fc69fd', 'request_id': 'req-1dc48195-536b-492d-b351-c36b565900e4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.517687 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '245af7b7ed61405ba444cddd26fc69fd', 'request_id': 'req-1dc48195-536b-492d-b351-c36b565900e4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.527102 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '245af7b7ed61405ba444cddd26fc69fd', 'request_id': 'req-1dc48195-536b-492d-b351-c36b565900e4'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.541669 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] The token cache is being invalidate because project 245af7b7ed61405ba444cddd26fc69fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:04:31.541939 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.542829 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.544791 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1445/2931] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:04:31 2022] DELETE /identity/v3/projects/245af7b7ed61405ba444cddd26fc69fd => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:04:31.549295 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:31.551390 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:31.566502 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b6f6b1-357e-4ba2-b215-1e6bea7bc691 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:31.627905 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:31.628600 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:31.628879 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:31.629185 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] PATH_INFO: `/v3/users/0aed6c0f3bea4362a4c78b7f4d0522fe` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:31.630352 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aed6c0f3bea4362a4c78b7f4d0522fe)` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:31.632275 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:31.683959 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aed6c0f3bea4362a4c78b7f4d0522fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.693046 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aed6c0f3bea4362a4c78b7f4d0522fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.693682 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aed6c0f3bea4362a4c78b7f4d0522fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.694553 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aed6c0f3bea4362a4c78b7f4d0522fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.702243 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.notifications [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aed6c0f3bea4362a4c78b7f4d0522fe'} {{(pid=93090) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:31.716306 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:31.733587 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1487/2932] 10.222.0.18 () {58 vars in 1224 bytes} [Tue Sep 13 20:04:31 2022] DELETE /identity/v3/users/0aed6c0f3bea4362a4c78b7f4d0522fe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:04:31.915485 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Authenticating user token {{(pid=93091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:31.916992 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.fernet_utils [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:31.932096 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc48195-536b-492d-b351-c36b565900e4 admin admin] Validating token access rules against request {{(pid=93091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:31.955389 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:31.995873 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc92c723f30741629d9940fdc108b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'af00e3aad1ad462b983823ea889f0033', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:31.996542 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:31.997457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:31.997457 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] PATH_INFO: `/v3/projects/936d970c33ff455983a5656d5e3c88c7` {{(pid=93091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:31.998361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=936d970c33ff455983a5656d5e3c88c7)` {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:32.000255 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] RBAC: Authorization granted {{(pid=93091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:32.061446 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936d970c33ff455983a5656d5e3c88c7', 'request_id': 'req-b0b23afd-52a0-4412-9513-4b03dc262bf2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:32.068052 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936d970c33ff455983a5656d5e3c88c7', 'request_id': 'req-b0b23afd-52a0-4412-9513-4b03dc262bf2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:32.075993 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936d970c33ff455983a5656d5e3c88c7', 'request_id': 'req-b0b23afd-52a0-4412-9513-4b03dc262bf2'} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:32.086744 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] The token cache is being invalidate because project 936d970c33ff455983a5656d5e3c88c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:04:32.087053 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:32.087710 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG keystone.notifications [None req-b0b23afd-52a0-4412-9513-4b03dc262bf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:04:32.089361 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: [pid: 93091|app: 0|req: 1446/2933] 10.222.0.18 () {58 vars in 1233 bytes} [Tue Sep 13 20:04:31 2022] DELETE /identity/v3/projects/936d970c33ff455983a5656d5e3c88c7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:04:41.734850 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:42.083510 dv-cinder-828806-6-iscsi devstack@keystone.service[93091]: DEBUG dbcounter [-] [93091] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:30.444661 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Authenticating user token {{(pid=93090) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:30.446331 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:30.605494 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bfb598-4557-44a8-84e5-1ac487dcb2f8 admin admin] Validating token access rules against request {{(pid=93090) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:30.669631 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4895ca8a2854fb38ce08ce74b628671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12814ce72efb432b9bfaa111d095abb9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93090) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:30.670255 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] REQUEST_METHOD: `GET` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:30.670535 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] SCRIPT_NAME: `/identity` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:30.670856 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93090) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:30.672926 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.fernet_utils [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93090) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:30.684770 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:30.686421 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40082a-4000-47b3-9ba5-803b2d83e826 service placement] RBAC: Authorization granted {{(pid=93090) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:30.696688 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: [pid: 93090|app: 0|req: 1488/2934] 10.222.0.18 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:40.692238 dv-cinder-828806-6-iscsi devstack@keystone.service[93090]: DEBUG dbcounter [-] [93090] Writing DB stats keystone:SELECT=89 {{(pid=93090) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}